Skip to content

fix: dbt microbatch parameter conversion#5298

Merged
eakmanrq merged 1 commit intomainfrom
eakmanrq/dbt_microbatch_fix_parameters
Sep 4, 2025
Merged

fix: dbt microbatch parameter conversion#5298
eakmanrq merged 1 commit intomainfrom
eakmanrq/dbt_microbatch_fix_parameters

Conversation

@eakmanrq
Copy link
Collaborator

@eakmanrq eakmanrq commented Sep 4, 2025

Fixes two misunderstanding of the dbt microbatch:

  • SQLMesh batch_size is always 1 for dbt microbatch. In other words, they never issue a query that spans multiple intervals.
  • concurrent_batches really determines, in SQLMesh terms, if a model depends on self which ultimately determine if the snapshot depends on past. Like SQLMesh, dbt determines this automatically by looking for self-referenes but users can force this with the concurrent_batches flag. Therefore if we see is False, meaning it should run sequentially, then we add the model to the depends_on.

@eakmanrq eakmanrq merged commit 58e3eca into main Sep 4, 2025
36 checks passed
@eakmanrq eakmanrq deleted the eakmanrq/dbt_microbatch_fix_parameters branch September 4, 2025 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants