Home/Insights/Data Engineering
Watch the video with chapters & transcript →
DATA ENGINEERING
In short: dbt (data build tool) transforms data already in your warehouse using plain SQL, adding version control, testing, and documentation — if you know SQL, you already know 90% of dbt.
dbt sits between the raw data in your warehouse and the reports your stakeholders see. Ani Björkström, a tech consultant based in Stockholm, explains where dbt fits in the pipeline, what life without it looks like, and when to skip it.
dbt sits between the warehouse, where source data lands raw and messy, and the presentation layer — Hex, Streamlit, or Power BI reports. Upstream, sources like CRM databases feed extract-and-load tools, which save data into a warehouse such as Snowflake, BigQuery, or Fabric.
dbt itself does not process your data. It is the brain: you write models, which are just SELECT queries in SQL, and the warehouse runs them — in Snowflake, your Snowflake warehouses execute the dbt models.
Through the ref function, which names the source each model reads from, dbt builds a dependency graph and knows which models depend on which. When you run dbt run, it respects that order — staging models like stage_orders and stage_customers first, then the enriched data mart, then downstream contracts like daily_revenue.
The graph also protects the team, not just the run order. Everyone can see that daily_revenue reads from orders, so a staging change immediately signals which downstream tables may need updating.
Without dbt, transformations live in stored procedures and SQL files on a shared drive, run by scheduled jobs. The setup works — until a new joiner cannot learn the execution order, and there is no version control, testing, or documentation.
This is not because data people are sloppy; SQL alone has no framework for dependencies, tests, or docs. dbt fixes each gap: models live in Git, tests like "order_id must be unique and not null" sit in a schema YAML file and run with dbt test, and dbt docs generates a searchable site covering every table, column, and linkage from raw sources to dashboards.
Use dbt when transformations are SQL in a cloud warehouse, more than two people work on the project, broken data has a real cost, and you have outgrown a handful of queries. Skip it in the cases below.
| Use dbt when | Skip dbt when |
|---|---|
| Transformations are SQL in a cloud warehouse | You are still choosing an extract-and-load tool — dbt only transforms data already loaded |
| More than two people work on the project | You are the only person on the team |
| Broken data has a real cost | Your work is mainly Python and machine learning |
| You have outgrown a handful of queries | You are not comfortable with Jinja or framework conventions |
No. dbt writes and orders SQL models, but the warehouse does all the compute when the models run.
No. dbt models are SELECT queries in plain SQL, so knowing SQL covers 90% of dbt; the rest is conventions like the ref function and schema YAML files.
You declare expectations in a schema YAML file and a single dbt test command runs all of them, instead of waiting for a stakeholder to report a broken dashboard.
Read the full video transcript, with chapters, on the video page →
Want this working inside your finance team?