Skip to content

chore: add dbt microbatch interface#5272

Merged
eakmanrq merged 1 commit intomainfrom
eakmanrq/dbt_adapter_microbatch_interface
Sep 2, 2025
Merged

chore: add dbt microbatch interface#5272
eakmanrq merged 1 commit intomainfrom
eakmanrq/dbt_adapter_microbatch_interface

Conversation

@eakmanrq
Copy link
Collaborator

@eakmanrq eakmanrq commented Sep 1, 2025

This PR does not add runtime support for microbatch - it just allows the dbt adapter to load microbatch models. Follow up PR(s) will be adding runtime support so it matches dbt behavior.

Comment on lines -259 to -264
for field in ("forward_only", "auto_restatement_cron"):
field_val = getattr(self, field, None)
if field_val is None:
field_val = self.meta.get(field, None)
if field_val is not None:
incremental_kind_kwargs[field] = field_val
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This used to add forward_only to the incremental_kind_kwargs when it should really be part of incremental_by_kind_kwargs. This PR fixes that.

disable_restatement = (
not self.full_refresh if self.full_refresh is not None else False
)
incremental_by_kind_kwargs["disable_restatement"] = disable_restatement
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disable_restatement is part of IncrementBy so it should be part of incremental_by_kind_kwargs and this PR adds it to it.

merge_filter = None
if self.incremental_predicates:
dialect = self.dialect(context)
incremental_kind_kwargs["merge_filter"] = exp.and_(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge_filter is not part of the incremental kind model so this PR fixes that.

@eakmanrq eakmanrq merged commit 70c8deb into main Sep 2, 2025
36 checks passed
@eakmanrq eakmanrq deleted the eakmanrq/dbt_adapter_microbatch_interface branch September 2, 2025 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants