Data Engineering

Explore the feasiblility of using Large Large Language Models to Document DBT Queries

dbt (Data Build Tool) is an open-source tool used in data engineering to help transform raw data in a data warehouse using SQL-based models. It acts as a T in the ETL (Extract, Transform, Load) or ELT (Extract, Load, Transform) process, where the data is first loaded into a warehouse and then transformed into usable datasets. Key Features of dbt SQL-Based Transformations – Uses SQL (with Jinja templating) to define transformation logic. Modularity & Reusability – Allows breaking down complex queries into smaller, manageable models. Version Control & Collaboration – Works well with Git for tracking changes. Automated Testing – Provides built-in testing for data quality. Dependency Management – Uses a DAG (Directed Acyclic Graph) to manage model dependencies. Documentation Generation – Automatically generates docs for models.

Somethings I did

• Onboarding
• Introduction
• Task description

Mid Presentation Presentation