DATABRICKS
type: databricksUnified analytics platform combining data engineering, data science, and business analytics on a lakehouse architecture.
DVT SUPPORT
read in federation models
materialize federation results
run full dbt projects against it
dvt seed --target loads here
DVT supports Databricksbecause both dbt and Sling do: dbt drives standard materializations when it's your default target, Sling moves data in and out during federation. Connectivity for both layers is checked with dvt debug --all.
PREREQUISITES
Driver: databricks-sql-connector — installed automatically by:
dvt sync
CONFIGURATION FIELDS
| FIELD | TYPE | REQUIRED | DEFAULT | DESCRIPTION |
|---|---|---|---|---|
| type | string | yes | — | Must be `databricks` |
| host | string | yes | — | Workspace URL (adb-xxx.azuredatabricks.net) |
| http_path | string | yes | — | SQL warehouse HTTP path (/sql/1.0/warehouses/abc) |
| token | string | yes | — | Personal access token |
| catalog | string | no | — | Unity Catalog name |
| schema | string | no | default | Default schema |
| threads | integer | no | 4 | Number of parallel threads |
PROFILES.YML EXAMPLE
my_project:
target: dbx_dev
outputs:
dbx_dev:
type: databricks
host: adb-123456.azuredatabricks.net
http_path: /sql/1.0/warehouses/abc123
token: "{{ env_var('DBX_TOKEN') }}"
schema: defaultSOURCES.YML EXAMPLE
sources:
- name: lakehouse
meta:
connection: dbx_dev
schema: bronze
tables:
- name: raw_events
- name: raw_usersDBT ADAPTER FEATURES
When Databricks is your default target, every standard dbt model runs through the official dbt adapter — all of its documented capabilities apply unchanged:
MATERIALIZATIONS
tableviewincrementalephemeralstreaming_tablematerialized_viewf_tablef_incrementalINCREMENTAL STRATEGIES
appendmergeinsert_overwritereplace_wheremicrobatchNOTABLE CONFIGS
- ▸Delta file format
- ▸liquid clustering
- ▸partition_by
- ▸tblproperties
- ▸Unity Catalog three-level namespace
FEDERATION INCREMENTAL STRATEGIES
Strategies available when this engine is the target of a f_incremental model: