Overview Quickstart CLI Commands .sql Models .py Models profiles.yml sources.yml Execution Paths

BIGQUERY

Google's serverless, highly scalable data warehouse. Pay-per-query pricing makes it cost-effective for intermittent analytics.

PREREQUISITES

Driver: google-cloud-bigquery — installed automatically by:

dvt sync

CONFIGURATION FIELDS

FIELD	TYPE	REQUIRED	DEFAULT	DESCRIPTION
type	string	yes	—	Must be `bigquery`
project	string	yes	—	GCP project ID
dataset	string	yes	—	BigQuery dataset
location	string	no	US	BigQuery location (US, EU, etc.)
threads	integer	no	4	Number of parallel threads

PROFILES.YML EXAMPLE

my_project:
  target: bq_dev
  outputs:
    bq_dev:
      type: bigquery
      project: my-gcp-project
      dataset: analytics
      location: US
      threads: 4

SOURCES.YML EXAMPLE

sources:
  - name: gcp_data
    connection: bq_dev
    database: raw_project
    schema: raw_dataset
    tables:
      - name: events
      - name: users

INCREMENTAL STRATEGIES

✓ Append✓ Delete+Insert✓ Merge

KNOWN LIMITATIONS

⚠Uses Application Default Credentials (ADC) — run `gcloud auth application-default login`