Skip to content

Fix!: stop treating dbt schema data types as columns_to_types#5231

Merged
georgesittas merged 4 commits intomainfrom
jo/do_not_use_dbt_data_type
Aug 27, 2025
Merged

Fix!: stop treating dbt schema data types as columns_to_types#5231
georgesittas merged 4 commits intomainfrom
jo/do_not_use_dbt_data_type

Conversation

@georgesittas
Copy link
Contributor

@georgesittas georgesittas commented Aug 26, 2025

After changing sushi_dbt as I did in this PR, the following commands run successfully and I can see unused_column in local.duckdb:

$ dbt -t duckdb seed
$ dbt -t duckdb run

Whereas SQLMesh fails to execute the project, because it currently treats the columns in schema.yaml as the source of truth, which does not match dbt's behavior:

$ sqlmesh plan  # Error: In model 'sushi.top_waiters', a description is provided for column 'waiter_id' but it is not a column in the model.

I verified locally (using dbt-duckdb) that setting data_type in schema.yml does not determine the actual DDL type. For example, for a simple model like SELECT 1 AS c, the following entry does not result in a TEXT type for c:

models:
  - name: simple_model
    description: "A simple model with a single column"
    columns:
      - name: c
        description: "A constant value"
        data_type: text

The data_type appears to mostly be relevant when contract is enforced, in which case it acts as an audit. For example, if I add the following config to the above model:

    config:
      contract:
        enforced: true

I see this error at runtime (dbt compile ran fine):

$ dbt run
...
22:52:33    Compilation Error in model simple_model (models/simple_model.sql)
  This model has an enforced contract that failed.
  Please ensure the name, data_type, and number of columns in your contract match the columns in your model's definition.

  | column_name | definition_type | contract_type | mismatch_reason    |
  | ----------- | --------------- | ------------- | ------------------ |
  | c           | INTEGER         | VARCHAR       | data type mismatch |

@georgesittas georgesittas requested a review from a team August 26, 2025 22:47
@georgesittas georgesittas changed the title Fix: stop treating dbt schema data types as columns_to_types Fix!: stop treating dbt schema data types as columns_to_types Aug 27, 2025
@georgesittas georgesittas force-pushed the jo/do_not_use_dbt_data_type branch from 932b188 to 8ffd8c5 Compare August 27, 2025 11:57
@georgesittas georgesittas force-pushed the jo/do_not_use_dbt_data_type branch 2 times, most recently from 47f42de to 9f17fa1 Compare August 27, 2025 18:22
@georgesittas georgesittas force-pushed the jo/do_not_use_dbt_data_type branch from 6664c5c to 9531e27 Compare August 27, 2025 21:00
@georgesittas georgesittas merged commit db58405 into main Aug 27, 2025
35 of 36 checks passed
@georgesittas georgesittas deleted the jo/do_not_use_dbt_data_type branch August 27, 2025 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants