Data Engineering
Overview
Worked on the Google Maps team to improve search result rendering performance.
Experience Details
Explore the feasiblility of using Large Large Language Models to Document DBT Queries
dbt (Data Build Tool) is an open-source tool used in data engineering to help transform raw data in a data warehouse using SQL-based models. It acts as a T in the ETL (Extract, Transform, Load) or ELT (Extract, Load, Transform) process, where the data is first loaded into a warehouse and then transformed into usable datasets. Key Features of dbt SQL-Based Transformations – Uses SQL (with Jinja templating) to define transformation logic. Modularity & Reusability – Allows breaking down complex queries into smaller, manageable models. Version Control & Collaboration – Works well with Git for tracking changes. Automated Testing – Provides built-in testing for data quality. Dependency Management – Uses a DAG (Directed Acyclic Graph) to manage model dependencies. Documentation Generation – Automatically generates docs for models.
Somethings I did
- Onboarding
- Introduction
- Task description