OverviewQuickstartdbt CompatibilityThe Style Guideprofiles.ymlsources.ymlConfigurationSupported Databases

DATA PROFILING

PORT :46105 · STARTS WITH dvt serve · PROFILE RUNS ANSWER LOOPBACK ONLY

Pick any relation — models by name, sources from any engine — and PROFILE: null counts, distinct counts, min/max and averages, computed live through federation. No dbt package, no adapter support matrix, nothing injected into your DAG: if DVT can federate it, DVT can profile it — every engine, every bucket, federated sources included.

Starting

dvt serve              # the suite, profiling included
dvt profiling          # just profiling (46105)
dvt kill -s profiling  # stop it

dvt docs generateis helpful, not required — a fresh catalog gives every column its type up front, but relations the catalog doesn't know get their columns discovered live from the relation itself.

What a profile tells you

  • null bars that shout — each column gets a null-percentage bar — amber past 10%, red past 50%. Dead columns are visible in one glance.
  • distincts, min/max, avg — distinct counts for every column; min/max for orderable types; averages for numeric ones. Complex types (structs, maps, JSON) honestly report nulls and distincts only — and LOBs (CLOB/BLOB), which refuse even COUNT(DISTINCT), report null counts alone.
  • one wide query, executed live — the whole profile is a single aggregate SELECT built by DVT — with a COPY button and the full text one click away, as always.
  • profiles are remembered — results cache per project (.dvt/profiles.json), so the app opens showing what you learned last time. RE-PROFILE is always live; CLEAR SELECTED forgets what you choose.

Where compute happens — the honest table

  • databases profile themselves — a database relation is single-connection, so the whole aggregate takes the engine's own direct path — computed BY Postgres, Snowflake, Oracle, wherever the data lives. One row crosses the wire. DuckDB is never in the loop.
  • files compute in DuckDB — CSV, JSON, Parquet, buckets — a file has no engine to ask, so Sling reads it once and DuckDB (the compute for things that don't have compute) does the aggregation.
  • columns are never a wall — the catalog supplies column names and types when it has them; when it's stale or silent — file sources, freshly added relations — the profiler asks the relation itself: a one-row sample through the same live path, types inferred from the answer. And if an engine rejects the catalog's spelling (mixed-case columns), the profiler discovers the true names and retries by itself.

Profiling in batch

  • select, then go — checkboxes on every relation with an ALL toggle; PROFILE SELECTED (LIVE) drains the selection four at a time — most relations live on different engines, so the concurrency is pure speedup.
  • dots tell the truth — grey never profiled, amber running, green profiled, red failed — with the engine's real error held right on the relation, not a generic excuse.
  • ■ STOP — halts the batch and aborts in-flight requests. Honestly noted: a query already running on a warehouse finishes there; DVT just stops waiting for it.
  • kind × storage badges — every relation states what it is: MODEL or SOURCE, TABLE or FILE — buckets and filesystem connections wear FILE.

Why there's no profiling package

dbt profiling packages support a handful of warehouse adapters and can only see the default target. DVT's profiler is built on the federation engine instead: the same machinery that reads a MySQL source and a Snowflake dimension in one model computes their statistics — so the support matrix is simply every adapter DVT has, and your project's DAG stays untouched.

← The App SuiteChat with Martin →