v0.2.59

ONE NAME, ONE TYPE

Two classes of drift closed by the same method — measure the whole matrix, then decide once in one place — plus the flip's one lane and the federated snapshot. Casing first: six rulings had fixed identifier case one hop at a time, each a fold predicted from a table meeting a real engine. A casing gauntlet measured an identifier corpus across every live engine, every way a relation is created and read (232 cells), and three laws replaced the predictions. Then types, the same way: a column crosses type systems twice on the federation path and every door re-inferred it — decimals widening to (24,6), timestamps with time zone landing as plain datetimes, Oracle booleans landing as strings, watermarks losing their microseconds. One type home now carries every column as Arrow, pins every column DVT can type on every engine, pins declared shapes on extraction, renders literals with precision, and documents the crossings an engine forces instead of fighting them. The metadata store gained DVT's own world: what every lane declared for every landed column and what the engine's catalog reports, verified after each load and exported as Landed and Mapping sheets. The flip has exactly one translator now — the AI lane, machine-judged, engine-compiled; the deterministic transpiler is gone. f_snapshot joins f_table and f_incremental as the federated type-2 snapshot. And the ring that judges all of it grew: client-owned fixtures, a source that moves between builds, hooks audited, landed types verified on every hop.

FIXED

▸A single-engine model on the sling direct path (the model's SQL run on the source, the stream landed straight on the target) landed a declared decimal(10,2) as decimal(24,6) on MySQL: that lane handed the loader no type pins. It now pins the source's declared shapes by the SQL's output names, aliases included, exactly as the federation pipeline pins its parquet.
▸dvt show on a default-target model went through the adapter's own preview wrapper, and on SQL Server that wrapper appends ORDER BY … FETCH straight onto the model, so an incremental model carrying a UNION ALL was refused. Every preview now runs through DVT's own subquery wrapper on the node's engine: dbt compiles the node (with DVT's per-target incremental state), DVT runs it with TOP, FETCH FIRST or LIMIT as the engine spells it.
▸When the bulk loader's own process crashed at start-up (a Go panic before the statement reached the engine), a DROP in dvt retract or a hook failed outright, although the next invocation always worked. The statement and query lanes now give a crashed loader one more try on every engine, as loads already did.
▸DROP, hooks, dvt exec, previews and the catalog read had no second chance on a cloud warehouse that was still waking: a retract on Databricks left three relations behind when the auto-stopped warehouse did not answer in time. The statement and query lanes now retry a waking Snowflake, Databricks or BigQuery connection on the same warehouse clock as loads.
▸A cloud warehouse that stayed silent past the driver's own deadline ("context deadline exceeded") failed the load outright, while the same silence spelled as a request error was retried on the warehouse clock. Both spellings now get the same second chance (60 s, then 120 s) on Snowflake, Databricks and BigQuery targets.
▸A cloud warehouse whose storage was throttling ("AWS S3 is throttling requests; try again later" from Databricks' Files API) failed a one-line query outright: loads rode the throttle out, but on the query lane the loader's process crashed on it, the one crash retry crashed the same way, and a throttle was not on the list of faces that earn the warehouse clock. It is now: after the crash retry, a throttled Snowflake, Databricks or BigQuery connection gets the same 60 s and 120 s window as a sleeping one.
▸dvt debug --all could report an installed adapter as "not installed" for one or two outputs at random: it checks every output in parallel and each check imports the output's adapter, and two adapters importing at the same moment could fail out of a half-initialised shared module. The check now imports one adapter at a time and gives a failure that is not a missing module one more try; a genuinely missing adapter is still reported with the dvt sync hint.
▸A flip to SQL Server left an Oracle-style FETCH FIRST 1 ROWS ONLY in place as "already valid": both readings of the file rendered the same T-SQL and every keyword survived, but SQL Server's FETCH needs an OFFSET and an ORDER BY the file never wrote, and the build failed. The neutrality judge now has a fifth reading: a clause the new engine's rendering adds to a file that lacks it is a translation the file still needs, so the file goes to the AI lane and comes back as TOP 1.
▸The flip's engine judge kept reporting installed, correct models as "needs a human" and reverting AI repairs on every flip: with nothing built on the new target yet, every judged model fails for a missing upstream, and the check that separates a missing upstream from bad SQL relied on a control relation that was simply the first source in the project, on this project an SFTP file no SQL engine can read. The control is now a source declared on the new target, so unbuilt upstreams are named as such and no repair round is wasted on them.
▸When a cloud warehouse refused work for the day (Databricks Free Edition: "you have hit your free daily limit, please come back again tomorrow"), every statement against it was retried on the warehouse-wake clock because the refusal's wording ends in "request error": a retract of twenty relations there ran past half an hour. A daily-quota refusal is now recognised as final and reported in seconds; sleeping warehouses still get their wake window.
▸A landed-types verification could still fail under a parallel run with a second DuckDB wording for the same open-mode race ("Unique file handle conflict … already attached by another handle"), and the note that should have recorded the failure hit the same conflict, so the landing read as having no catalog answer. The metadata store's connect discipline now recognises that wording too: a reader reads through the read-write handle, a writer waits out the short-lived reader.
▸A model with a custom schema landed correctly on Databricks but its landed-types verification reported every column as not found: Databricks stores schema and table names in lower case and its information_schema matches that stored spelling exactly, while DVT asked for the schema as dbt had spelled it (default_STG). The catalog read now asks Databricks in its own spelling; column names keep their case and match case-insensitively as before.
▸A single dropped staging upload on Databricks (the PUT of a parquet file into the staging Volume answering "failed to execute query") failed a model outright and skipped everything downstream; the load door retried sleeping, throttled, and crashed engines but not that. A failed staging upload on a cloud target is now retried on the same warehouse clock as a waking warehouse; SQL refusals and local targets are untouched.
▸dvt metadata refresh could fail on Databricks after its catalog query had already answered: the loader deletes its temporary export from the staging Volume afterwards, and the Files API dropped that delete once. Files API failures, upload or delete, are now retried on the warehouse clock on the query, statement and load lanes alike.
▸A flip to Databricks could write a model whose Jinja control blocks ({% if is_incremental() %} and friends) were wrapped in backticks: the AI lane quoted them like columns, the judges hold Jinja out and saw valid SQL, and the build met an empty backtick pair. Identifier quotes around a Jinja statement are now removed from every proposal before it is judged.
▸An engine's bulk accelerator that is installed but cannot run on the machine (Oracle's sqlldr after a macOS upgrade dropped Rosetta: "Bad CPU type in executable") failed every load to that engine, while dvt sync still reported the accelerator present. A load whose accelerator cannot start is now re-issued once on the loader's plain lane with a warning naming the fix, and dvt sync executes each accelerator it finds and reports one that cannot run as missing.
▸The loader's occasional lost-column-name transient (retried once by name on every engine since 0.2.59's ring) had no Oracle spelling: an incremental load on the plain lane failed on ORA-01741 (illegal zero-length identifier) with no second try, one invocation after the bulk accelerator fell over. Oracle's spelling is recognised, and a load re-issued on the plain lane gets the same one retry for the same transients as a first run.
▸A retried load could double its rows: the loader's merge lane fills DVT's staging table first and merges afterwards, and a run that died in between left its rows there for the retry to append to, so the merge landed both copies. Every retry the load door makes now drops the load's staging table first.
▸An extraction from a local engine could fail on the loader's own internal data pipe (its embedded DuckDB listens on a local port it picks; when another app on the machine holds that port, the loader's own request lands on the wrong server) and DVT reported the source as having failed, with no retry because local sources were held to fail deterministically. That face is recognised as the loader's, not the engine's, and gets one retry on every lane for every engine.
▸The federation results DVT merged into run_results.json lacked three fields dbt's own reader requires, so any dbt command that reads the artifact back (dvt clone --state, result: selectors) refused it. The merged entries now carry dbt's full result shape and a test reads them back with dbt's class.
▸After a federation run, target/run_results.json named the run's internal pre-compile as the command that ran, so dvt retry re-ran a compile and reported success while landing nothing. The artifact now names the verb you ran (run or build).
▸On a DuckDB or SQLite warehouse file, a federated model's landed-types verification (an in-process read) could hold the file for the instant the loader was landing a sibling model into it, and the loader fails fast on a held lock — one hop's build #2 lost a Python model to a race between two of DVT's own lanes. Both lanes now take the file's one lock, and a file held by another process is waited out on a bounded clock before the loader gives up.
▸A tz-aware watermark could be pushed in the machine's time zone: a {{ this }} index read back from Databricks (where DVT lands timestamps as the tz-aware Delta TIMESTAMP) came out of DuckDB in the session zone (Asia/Riyadh on the ring's machine), the bound was rendered as that wall clock with the offset dropped, and three of ten delta rows fell under it. Bounds now render in UTC and the federation cache's session zone is UTC on every machine.
▸The extraction lane quoted mixed-case names that DVT itself had landed upfolded — the demo's 002003 reappearing one lane over on Snowflake and Oracle. DVT's own relations now render by the landing convention on every engine; the Oracle-only special case became the rule.
▸Reserved-word relation names (select, order, date…) rendered unquoted in hooks, extraction and retract on every engine except the postgres family — syntax errors, and retract could not even drop them. Every lane now quotes them, on the folded spelling, so the quoting can never change what resolves.
▸A relation name with a space crashed the bulk loader outright — a segmentation fault before any message, on all ten engines, every time. DVT now refuses such a name before loading (DVT066) and tells you to rename the model or seed, or set alias=. Filed upstream. The refusal judges the object's schema and name only — a duckdb or sqlite connection's file path is not a name, and the first ring stop of this release caught it judging one.
▸A table you quoted yourself with lowercase or mixed case was invisible to DVT's convention-based reads. Foreign relations are now read exactly as the metadata store — the engine's catalog — spells them, always quoted; the fold rules remain only for sources the store has never seen.
▸DVT's create-schema statement was invalid T-SQL on SQL Server and Fabric, and the executor swallowed the refusal — every custom-schema load there had been relying on the loader's own ensure without anyone knowing. Valid T-SQL now, and a refused ensure is reported.
▸A source's declared decimal arrived in DuckDB as DECIMAL(24,6) — the loader's default parquet decimal — and would have landed downstream with the widened declaration. The extraction door now pins the source's declared precision, scale and string widths on the parquet DVT reads, from the metadata store's facts. Measured: decimal(10,2) in, decimal(10,2) out.
▸A timestamp with time zone was pinned as a plain datetime on every engine, so the instant landed but its tz-awareness did not. It is pinned as a tz-aware type where the loader writes one faithfully (Postgres, Oracle, ClickHouse, Trino, DuckDB). On SQL Server, MariaDB and MySQL 5.7 the loader's own tz-aware write was measured to shift the instant by the machine's UTC offset — 07:11 UTC landed as 10:11 — so on SQL Server and the whole MySQL family DVT writes the UTC wall clock itself into datetime2(6)/DATETIME(6): the instant is exact, the awareness lost is a documented crossing in the landed world.
▸A config whose hook string carried nested Jinja — pre_hook="insert into {{ target.schema }}.audit …" — was torn in half by the flip: every Jinja regex stopped at the first }}, the AI lane was handed half a config, and the model came back with its config commented out. The flip (and config-checkup's config reader) now read Jinja the way Jinja does — strings whole, brackets balanced — so hooks with nested expressions travel intact.
▸On Postgres the loader created every decimal as bare numeric, pinned or not: the values were exact, the declared precision and scale were gone, and a round trip read them back as DECIMAL(24,6). DVT restores the declared numeric(p,s) with one ALTER right after the load (SQL Server, MySQL, Oracle, ClickHouse, Trino and DuckDB keep it themselves).
▸On Trino a UUID column made the loader create a uuid table column and then feed it a varchar(36) value — the whole load refused. UUIDs are declared varchar(36) on Trino, and the file and table agree.
▸Under a parallel run, a model's extraction could silently go unpinned and its landing silently unverified: DuckDB refuses to open the metadata store read-only while another thread holds it read-write in the same process, and that one error carried no "lock" for the store's retry to see. The store now reads through a read-write handle when a writer is present and waits for a short-lived reader when writing; a verification that fails is a warning with its reason written into the store, and an unpinned extraction is named in the log.
▸The flip's neutrality judge called a model with LIMIT 1 OFFSET 0 "already valid" for SQL Server, because SQLGlot renders both the DuckDB reading and the T-SQL reading as the same OFFSET … FETCH — the file on disk still said LIMIT and the build failed. The judge now has a fourth reading: any keyword the new dialect's rendering no longer uses (LIMIT, ILIKE, ::, ||, CAST … AS INTEGER on MySQL, CURRENT_DATE on T-SQL) sends the file to the AI lane.
▸A flip could change the case of a column name and nothing noticed until ClickHouse: the AI lane's Oracle translation upper-cased a model's bare identifiers (category → CATEGORY), Oracle and then MariaDB folded the difference away, and on ClickHouse — the one engine that keeps an unquoted name exactly as written — the model was judged already valid, left alone, and failed its build against the lowercase column its upstream lands. Two rules now: a translation is re-spelled to the author's own identifier case before it is written, and on ClickHouse a native model's bare column names take the spelling they landed with upstream (the ref'd model's projection, or the seed's header), with the flip ledger saying so.
▸The flip's engine judge sent a literal EXPLAIN to every engine; SQL Server has no such statement and Oracle refuses it, so on those two engines every native model "failed compile" and every AI repair was reverted. The judge now speaks each engine's own validation statement — EXPLAIN, EXPLAIN PLAN FOR on Oracle, sp_describe_first_result_set on SQL Server and Fabric — for the models and for its relation probe.
▸A seed with a boolean column landed on Oracle as VARCHAR2 "true"/"false": the result shaping that lands NUMBER 0/1 for federated models never reached the seed lane. Every non-parquet seed is converted through DuckDB already, so the seed door now shapes the parquet for the target the same way, and the landed world records the seed's own type beside the shaped pin.
▸A column named with a space landed on Oracle as COLUMN_1 and the landed world could not find it in the catalog: the loader lands names under the target's casing option and Oracle's upper fold had not been measured. It is measured and mirrored now (column 1 → COLUMN_1, MixedCase → MIXEDCASE), and every landing declares its column names through the casing home.
▸A model reading another DVT model on the same engine could re-land it with a different type: on Oracle a float lands as a bare NUMBER, the loader reads a bare NUMBER back as decimal(38,6), and the next model landed NUMBER(38,6), with digits at risk. Reading DVT's own landing now pins every column with what its model declared, wherever the landed world verified the engine kept it, so a same-engine round trip reproduces the first landing.
▸Databricks, measured the day the workspace came back: the loader's volume lane panicked on a parquet UUID column and the whole load died on it, so UUIDs are shaped to text in the parquet before the load, as on Trino. Every decimal lands DECIMAL(24,6) there whatever the pin — values exact, the declared precision a lost fact — which is now a documented crossing when the declared shape fits, and DVT restores the declared precision when it reads its own table back. Delta's TIMESTAMP is read as the tz-aware type it is.
▸After a flip, every f_incremental model whose local index survived from the old target compiled as incremental on the new one, pushed a delta bound, and merged zero rows into a table that had never been created there; the build reported OK. Incremental state is now a property of the model and the target together: DVT's own ledger says where each model was landed, an index without a landing on the current target is a first run there, and a first run drops the stale index so the next run rebuilds it from that target.
▸An f_snapshot on MySQL closed and reopened every row on every build (5 rows became 10, 110 became 207) while its own report said "0 changed". The snapshot read its standing versions back untyped, so a decimal(10,2) returned as DECIMAL(24,6) and 12.340000 no longer hashed equal to 12.34; and the report hashed nothing for check_cols='all'. The read-back now takes the same declared-shape pins every source takes, from the target's own catalog, and the report uses exactly the apply's predicate. On Oracle the same snapshot doubled once more because a DATE reads back as a timestamp (a documented crossing), so the diff now casts both sides to the model's own column types before comparing — the engine's spelling of a value can never masquerade as a change.
▸The bulk loader's own staging DDL intermittently lost the FIRST column's name — create table … ("" text, …), "zero-length delimited identifier" — on Postgres, MySQL, Trino and DuckDB, and the same lost name reappeared as a nameless struct_pack argument when the loader wrote a parquet to Azure; always on a first invocation and never on the next. Every cause on DVT's side was tested and ruled out live (the parquet's names, the CSV's byte-order mark, table state, load mode, pins), so these exact messages join the named transient classes that get one retry, on the table lane and the bucket lane alike; a real error still surfaces unchanged.
▸A federated model's pre-hook on a DuckDB target could fail with "Conflicting lock is held" when the loader's own DuckDB was landing a sibling model into the same file at that instant — 70 models in parallel, one race no user wrote. In-process opens of a file database now wait, bounded, for a lock another process holds, and a lock that never clears still fails with the engine's own words.
▸A boolean landed on Oracle as VARCHAR2 'true'/'false', so a hook's where flag = 1 failed. It lands NUMBER 0/1 — Oracle's own idiom — shaped in the result before the load.
▸Incremental watermark bounds were rendered to the second, so > '…12' re-read every row inside the truncated second on each append run. Bounds keep their microseconds; Oracle's format model says FF6.
▸A pandas column of integers with a single null landed as double (1 → 1.0). Python models now repair exactly that promotion — a float column with a null whose non-null values are all integral becomes a nullable integer — and say so; a null-free float column is left as written.
▸config-checkup judged a source by its table name where parse judges its identifier, so a client table declared under an alias read as DVT's own output. It judges the identifier now. And the flip's ledger printer had no mark for a manual-tier finding and exited 1 after a flip that had landed.
▸Strings on Postgres: the loader's generic string cast lands character varying(65500) while its text cast lands unbounded text — measured on the type gauntlet — so unbounded strings pin text on the Postgres family and can never be narrowed.

ADDED

▸The dbt verbs that must speak federation, implemented instead of passed through: dvt show previews a federated model from the relation it landed on its own engine (the sling direct path included) and runs an inline query over federated refs there; dvt source freshness measures a source declared on a non-default connection on that connection and merges it into sources.json; dvt clone names every federation model and skips it rather than leaving a broken pointer view; dvt retry re-runs the previous command's failed and skipped nodes through DVT's own dispatcher, federation lane included; dvt compile carries the per-target incremental state so an f_incremental compiles the branch it would run.
▸The ring battery grew from 32 to 48 stops per hop: version, sync, deps, debug, debug --all, list, compile (cold and warm), fix-warnings --check, source freshness, show, run-operation, retry, clone, clean-bak-files and clean — the whole dbt command surface, every hop, federated models included.
▸The type home (dvt/federation/types.py): Arrow as the carrying vocabulary; a native-to-Arrow reader for every engine's catalog spelling; Arrow-to-loader casts and Arrow-to-DDL maps per engine; a load door that pins every column DVT can type verifiably on every engine and names the rest; typed literals with precision, routed through the compensation home; documented engine crossings (Oracle's NUMBER and DATE, ClickHouse's DateTime64, MySQL's tz) judged faithful with the reason attached.
▸DVT's own world in the metadata store: every landed relation with, per column, the parquet type, what DVT declared to the loader, the DDL class it expected and what the engine's catalog holds — verified after every load by every landing lane (federation compute, the direct push-down, seeds, the dbt phase) and removed by retract. dvt metadata status prints it and names every unfaithful column; the export gains Landed and Mapping (the source-to-target mapping document, by relation lineage and column name). Columns carries the parsed facts — Arrow reading, length, precision, scale — beside every native type.
▸A catalog lane of DVT's own, and it speaks first: DVT reads each engine's catalog with its own SELECT per engine — the type as the engine spells it, precision and length included — and falls back to the loader's discover only where it has no SELECT (discover normalises numeric(10,2) to numeric, and its query uses a CTE MySQL 5.7 cannot run). DuckDB and SQLite files are read in-process, never through a subprocess that would wait on the file's lock.
▸f_snapshot — the federated type-2 snapshot, a model materialization beside f_table and f_incremental: DuckDB SQL over sources on any engines, strategy timestamp or check, dbt's four meta columns declared and pinned on every engine, the standing snapshot's open versions read back as typed parquet, new and changed versions opened and replaced versions closed in one merge on dbt_scd_id. First run creates, later runs merge; nothing changed applies nothing; no native pair, so it stays federated through every flip.
▸The type gauntlet: a corpus of types × every live engine × four lanes — load, round trip, literal, and the VALUE each engine renders back itself (instants in UTC, through the engine's own functions, never a client driver's conversion) — one pytest cell each, results persisted as JSON. Every pin and every documented crossing in the type home was admitted by one of its green cells; the values lane exists because a type can be faithful while the data behind it is wrong, and it found exactly that.
▸The casing gauntlet: an integration matrix of identifier corpus × live engine × create lane × read lane, one pytest cell each, results persisted as JSON — the regression net every future casing change must pass.
▸Python models take pyarrow Tables, polars frames, pyspark 4 DataFrames and DuckDB relations natively (Arrow is the interchange); pandas remains supported with the null-promotion repair.
▸The ring grew: client-owned fixture tables planted by the harness on every engine (a DVT output never re-enters a project as a source — ruling 407 holds on the ring's own project now), a delta table the harness moves between build #1 and build #2 so every incremental lane and both snapshot strategies are judged on real movement, pre- and post-hooks on native and federated models audited on every target, and a landed-types stop that reads DVT's own world back and demands every column faithful.

CHANGED

▸dvt flip-target-to has exactly one translator: the AI lane. The deterministic SQLGlot transpiler and its span transpiler are gone; what stays deterministic is what never lied — the role map, target provisioning, atomic writes with .baks, parse-after-flip, the compile and EXPLAIN judge, config-checkup and config-fix before and after. A neutrality judge reads each file as the old and the new dialect and compares the two readings (through the compensation home), so a file already valid on the new engine is left untouched with no AI spent; every other file gets its own bounded, validated conversation; a file the lane cannot carry is kept as written and named, then the same lane is asked for DuckDB SQL so the model can run federated instead of stranded. The engine judge now compiles and explains the untouched files too. --agentic is accepted and does nothing; --no-ai moves the target and the roles, judges, and names every file still to translate. Round-trip byte-identity across engines is no longer a promise — a translation is not a normal form; the ledger, the .baks and the engine judge are what you read.
▸Every seed is converted to parquet before loading on every engine, so every seed column can be typed; the strict-typing engine set is gone because every engine is strict now.
▸Oracle upfolds DVT's own relation names at the naming home (the extraction lane had been quoting mixed case past a landed MIXEDCASE); Trino downfolds; the postgres family quotes as written; ClickHouse stores what it is given. One convention per engine, decided once.
▸Retract's declared-source keep stays (a declared source is the client's, never dropped) but the ring no longer relies on it to protect DVT-landed fixtures; those are client tables the harness owns.
▸The parked engines: Snowflake (trial ended), Databricks (workspace inactive — and with it the simulated Snowflake lane), plus sqlite and StarRocks from campaign 1. Their fixtures are disabled behind one variable (cloud_lanes) and return when the lanes do.

Get it: pip install dvt-core==0.2.59 · wheels on PyPI for macOS (arm64 + x86_64) and Linux x86_64, Python 3.10–3.14 — the full platform matrix.

← v0.2.58 v0.2.60 →