Skip to content

dbt error patterns

How ODS recognises what dbt, and the engines dbt runs on, say when a node or a test fails (ADR-0025). This page is the dbt error catalogue (ods-provider-dbt::error_catalogue) at catalogue version 8. A test keeps it in step with the code.

A pattern reads only the error's redacted summary: its kind and its first line, with quoted values, numbers and SQL removed (ADR-0024), so a phrase can never match on, or show, a value. The message is lowercased before matching. Kind is the error's kind, which is either the engine's own (Binder Error) or the one dbt's header gave around it (Database Error); any means any kind. The first pattern whose kind and phrases all match wins. A space at either end of a phrase matters and is shown as ␠: column␠ matches column "x", not columnar.

Recorded from real dbt: yes when a message captured from a real run matches it: dbt 1.10, 1.11 or 1.12 on DuckDB (fixtures/dbt/jaffle-ods/artifacts/dbt-<version>-errors), or dbt-databricks on a Databricks SQL warehouse (dbt-databricks-errors, captured by the databricks workflow). Otherwise the pattern is taken from the public source named; a test still reaches it with a written message.

Pattern Symptom Kind The message holds Source Recorded from real dbt
dbt-undefined-macro unknown_macro any is undefined and macro that does not exist dbt-core's messages yes
dbt-missing-ref missing_ref any depends on a node named and which was not found dbt-core's messages yes
dbt-packages-not-installed packages_missing any specified in packages.yml, but only and installed in dbt-core's messages yes
dbt-packages-expected packages_missing any based on packages specified in packages.yml, but found only and installed in dbt-core's messages yes
dbt-profile-not-found profile_not_found any could not find profile named dbt-core's messages yes
dbt-target-not-found profile_not_found any does not have a target named dbt-core's messages no: from the source named
dbt-template-unexpected template_syntax compilation error unexpected␠ dbt-core's messages yes
dbt-template-expected-token template_syntax compilation error expected token dbt-core's messages no: from the source named
dbt-template-unknown-tag template_syntax compilation error unknown tag dbt-core's messages no: from the source named
dbt-python-model python_exception any python model failed dbt-core's messages yes
dbt-test-failed test_failed any configured to fail if dbt-core's messages yes
postgres-column-missing missing_column database error column␠ and does not exist PostgreSQL's documented messages no: from the source named
postgres-schema-missing missing_schema database error schema␠ and does not exist PostgreSQL's documented messages no: from the source named
postgres-function-missing missing_function database error function␠ and does not exist PostgreSQL's documented messages no: from the source named
postgres-relation-missing missing_relation database error relation␠ and does not exist PostgreSQL's documented messages no: from the source named
postgres-permission-denied permission_denied database error permission denied for PostgreSQL's documented messages no: from the source named
postgres-statement-timeout query_timeout database error canceling statement due to statement timeout PostgreSQL's documented messages no: from the source named
postgres-invalid-input type_mismatch database error invalid input syntax for type PostgreSQL's documented messages no: from the source named
postgres-password credentials_missing database error password authentication failed PostgreSQL's documented messages no: from the source named
postgres-unique constraint_violation database error violates unique constraint PostgreSQL's documented messages no: from the source named
postgres-not-null constraint_violation database error violates not-null constraint PostgreSQL's documented messages no: from the source named
postgres-connect warehouse_unavailable database error could not connect to server PostgreSQL's documented messages no: from the source named
spark-unresolved-column missing_column any [unresolved_column Apache Spark's error-conditions.json yes
spark-table-or-view-not-found missing_relation any [table_or_view_not_found] Apache Spark's error-conditions.json yes
spark-cast-invalid-input type_mismatch any [cast_invalid_input] Apache Spark's error-conditions.json yes
spark-datatype-mismatch type_mismatch any [datatype_mismatch Apache Spark's error-conditions.json yes
spark-check-constraint constraint_violation any [check_constraint_violation] Apache Spark's error-conditions.json no: from the source named
spark-not-null-constraint constraint_violation any [not_null_constraint_violation] Apache Spark's error-conditions.json no: from the source named
delta-not-null-constraint constraint_violation any [delta_not_null_constraint_violated] Delta Lake's delta-error-classes.json no: from the source named
delta-check-constraint constraint_violation any [delta_violate_constraint_with_values] Delta Lake's delta-error-classes.json no: from the source named
delta-new-not-null-constraint constraint_violation any [delta_new_not_null_violation] Delta Lake's delta-error-classes.json yes
delta-new-check-constraint constraint_violation any [delta_new_check_constraint_violation] Delta Lake's delta-error-classes.json yes
spark-schema-not-found missing_schema any [schema_not_found] Apache Spark's error-conditions.json no: from the source named
spark-unresolved-routine missing_function any [unresolved_routine] Apache Spark's error-conditions.json yes
delta-concurrent-write lock_conflict any [delta_concurrent_ Delta Lake's delta-error-classes.json no: from the source named

Besides these, a Python exception's name given as the error's kind (KeyError: …) is recognised as python_exception (python-exception). Only Python's built-in exceptions are ever named.

Warehouse plugins' patterns

A warehouse's own messages are its plugin's (ADR-0031 §3a). On a project on that warehouse, ODS asks the plugin's catalogue first, then the one above. On any other project, it is asked after the one above, so a run from before dbt named the warehouse, or from before the project moved, is still explained. Either way dbt's steps are offered, and the explanation names the catalogue that recognised the error.

The built-in Databricks plugin's catalogue (ods-provider-databricks::error_catalogue), at catalogue version 1, holds dbt-databricks's own messages (Apache-2.0, 1.12). Its patterns read the message only, whatever its kind. Each is taken from that source, and a test reaches it with a written message:

Pattern Symptom The message holds
databricks-cluster-start warehouse_unavailable error starting cluster
databricks-cluster-status warehouse_unavailable error getting status of cluster
databricks-connection warehouse_unavailable failed to create connection
databricks-command-timeout query_timeout command execution timed out
databricks-python-timeout query_timeout python model run timed out
databricks-oauth-required credentials_missing is required when not using access token
databricks-client-id-required credentials_missing is required to connect to databricks when and is present

The built-in DuckDB plugin's catalogue (ods-provider-duckdb::error_catalogue), at catalogue version 1, holds DuckDB's error kinds and messages (MIT), as dbt-duckdb reports them. Unlike the Databricks patterns, most need the error's kind (Binder Error, Catalog Error), as above. Recorded from real dbt: yes when a message from the recorded DuckDB runs matches it; otherwise it is from DuckDB's error kinds, and a test reaches it with a written message:

Pattern Symptom Kind The message holds Recorded from real dbt
duckdb-values-list-column missing_column binder error does not have a column named yes
duckdb-referenced-column missing_column binder error referenced column and not found yes
duckdb-table-missing missing_relation catalog error table with name and does not exist yes
duckdb-view-missing missing_relation catalog error view with name and does not exist no: from DuckDB's error kinds
duckdb-schema-missing missing_schema catalog error schema with name and does not exist no: from DuckDB's error kinds
duckdb-function-missing missing_function catalog error function with name and does not exist yes
duckdb-conversion type_mismatch conversion error (any message) yes
duckdb-constraint constraint_violation constraint error (any message) no: from DuckDB's error kinds
duckdb-dependent-entries dependent_objects dependency error because there are entries that depend on it yes
duckdb-write-conflict lock_conflict transactioncontext error conflict no: from DuckDB's error kinds
duckdb-file-lock lock_conflict any could not set lock on file no: from DuckDB's error kinds
duckdb-permission permission_denied permission error (any message) no: from DuckDB's error kinds
duckdb-interrupted query_timeout interrupt error (any message) no: from DuckDB's error kinds

Deliberately not recognised

  • An error that resembles a symptom without being it gets no pattern, or a symptom of its own: a missing schema is missing_schema, not a missing table, and a missing SQL function is missing_function, not an undefined macro. A pattern that named the wrong symptom would let unrelated evidence confirm the wrong cause (rule 3).
  • A table in a schema that doesn't exist, on Databricks: the warehouse reports the table ([TABLE_OR_VIEW_NOT_FOUND]), not the schema, so ODS says missing_relation, as the warehouse did, rather than guess which part is missing.
  • PostgreSQL's cannot drop … because other objects depend on it: the summary removes drop … as SQL, so the phrase never reaches the catalogue.

An error no pattern recognises is not_recognised. It takes the category its kind implies (dbt's header kind when the message's own says nothing), and ODS doesn't guess a cause.