In our previous post we described how to get source data into the Databaas Catalog with dlt. Landing the data is only step one: the three sources in that post each land on their own schedule, in their own table. In this post we describe how we can answer a business question by combining and transforming the data with dbt, running on Trino and orchestrated by Airflow with Cosmos, using the same example repo, databaas-dlt-dbt-demo. dbt is not the only way to transform data on the platform, but we recommend it here for its ease of use.
Why dbt
dbt is a transformation framework: you write plain SQL select statements, and dbt turns them into modular, tested models.
It works out the order the models have to run in, creates the views and tables, and runs the tests you define on them.
You only write the logic of the transformation; dbt takes care of the rest.
A dbt project on Databaas
There is nothing Databaas-specific about the dbt project itself: a dbt_project.yml, a profiles.yml and a models/ tree, a standard dbt-trino project layout.
In the example repo the models follow the usual layers: staging models clean each source, intermediate models keep the latest reading per hour, and a mart combines the marine and wind forecasts with the KNMI observations into one table, sea_state_hourly.
What is specific to Databaas is where dbt writes.
The Trino catalog is always iceberg, and what dbt calls a schema is a namespace in the Databaas Catalog.
A development copy of a pipeline should never write into production’s namespace, so the example repo keeps the base names of its namespaces in a small JSON file next to the project:
{
"catalog": "iceberg",
"namespaces": {
"TRINO_NAMESPACE": "marine_transformed"
}
}
Before dbt runs, each name is passed through scoped() from databaas-utils, and handed to dbt as an environment variable:
config = json.loads((DBT_PROJECT_DIR / "dbt_config.json").read_text())
namespaces = {env: scoped(base) for env, base in config["namespaces"].items()}
In production scoped("marine_transformed") returns the name as it is; on a branch it returns marine_transformed_dev_<branch>.
The same dbt project can then run for production and for every open pull request without them overwriting each other.
Connecting dbt to Trino
Trino on Databaas only accepts a Databaas access token, over HTTPS.
dbt-trino supports that with the jwt method, so the profile reads the token from an environment variable:
method: jwt
jwt_token: "{{ env_var('TRINO_JWT') }}"
host: "{{ env_var('TRINO_HOST', 'trino.example.databaas.app') }}"
http_scheme: https
catalog: "{{ env_var('TRINO_CATALOG', 'iceberg') }}"
schema: "{{ env_var('TRINO_NAMESPACE', 'marine_transformed') }}"
In Airflow the token belongs to the pipeline’s own service account, the same identity the dlt tasks use to write to the Catalog.
dbt can therefore read and write exactly what that account has been granted, and nothing more.
On your laptop you run the same project as yourself, with a token from the databaas command line tool; see laptop login for how to sign in.
Running dbt with Cosmos
The simplest way to run dbt from Airflow is one task that calls dbt’s Python API, dbtRunner.
The downside is that to Airflow the whole dbt project is then one task: when one model fails, you rerun all of them.
Cosmos turns a dbt project into an Airflow task group instead, with every model and test as its own task; the Databaas docs describe how to set it up. When one model fails, you rerun only that model and the models that depend on it. In the example repo every DAG builds its group with one shared function:
DbtTaskGroup(
group_id=group_id,
project_config=ProjectConfig(
dbt_project_path=DBT_PROJECT_DIR,
manifest_path=MANIFEST_PATH,
env_vars={"TRINO_CATALOG": config["catalog"], **namespaces, **source_env},
),
profile_config=ProfileConfig(..., profile_mapping=LakehouseTrinoJWTProfileMapping(...)),
execution_config=ExecutionConfig(execution_mode=ExecutionMode.WATCHER),
render_config=RenderConfig(load_method=LoadMode.DBT_MANIFEST, select=select),
)
Two settings matter here.
ExecutionMode.WATCHER runs a single dbt build for the whole selection and lets each model’s task report its own result, so dbt starts once instead of once per model.
LoadMode.DBT_MANIFEST builds the task graph from dbt’s manifest.json instead of calling dbt: Airflow reads the DAG files where dbt isn’t installed.
That manifest is generated with dbt parse in CI, before the DAGs are validated and deployed; see develop and ship a pipeline for that workflow.
Cosmos also needs the token for Trino. Its built-in Trino mapping reads a fixed token from an Airflow connection, but a Databaas token expires within the hour. The example repo extends the mapping so it asks databaas-utils for a fresh token every time a task runs:
class LakehouseTrinoJWTProfileMapping(TrinoJWTProfileMapping):
airflow_connection_type = "databaas_lakehouse"
def get_dbt_value(self, name: str):
if name == "jwt_token":
from databaas_utils.airflow import get_settings
return get_settings(self.conn_id).auth_header().removeprefix("Bearer ")
return super().get_dbt_value(name)
The token is fetched inside the task, not when Airflow reads the DAG file, so there is a fresh token at the start of each run.
Waiting on Assets: the marts DAG
In the previous post each ingestion DAG emits an Asset once its data has landed. In the example repo, each source DAG now also runs its own dbt models in a Cosmos group, and emits its Asset when that group is done.
The mart that combines all three sources gets a DAG of its own and it is scheduled on the Assets of all three sources:
schedule=AssetOrTimeSchedule(
timetable=CronTriggerTimetable("@daily", timezone="UTC"),
assets=(MARINE_FORECAST_ASSET | WIND_FORECAST_ASSET | KNMI_OBSERVATION_ASSET),
)
The | means the marts DAG runs as soon as any one source has new data, so a stalled source never holds back the others.
The daily timetable is a fallback: the mart still runs at least once a day, even when no source delivers new data.

In Airflow the marts DAG’s group shows the same lineage as dbt: the three staging models, the intermediate models with their uniqueness tests, and sea_state_hourly waiting on all three tests.
When one test fails, you see which one directly in the graph.
Build it yourself
The full pipeline from this post, ingestion and transformation, is in databaas-dlt-dbt-demo.
To build your own, start with the Databaas docs on dlt and dbt and on how to develop and ship a pipeline and the databaas-utils docs cover the helpers and the databaas command line tool.
For other ways to render and run a dbt project in Airflow, see the Cosmos docs, and if you build with Agents, the databaas-skills repo has skills for building Airflow pipelines on Databaas with dlt and dbt.
With the transformed data in the Databaas Catalog, you can now answer business questions, either by querying it directly from a notebook or SQL Lab, or by building visualisations on top of it.




