Skip to content

Fix: Variable overrides for dbt packages #5376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
izeigerman merged 1 commit into from
Sep 15, 2025
Merged

Fix: Variable overrides for dbt packages #5376

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions sqlmesh/dbt/project.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ def load(cls, context: DbtContext, variables: t.Optional[t.Dict[str, t.Any]] = N
package.variables.update(package_scoped_vars)
else:
package.variables.update(all_project_variables)
if variable_overrides:
package.variables.update(variable_overrides)

Comment on lines +116 to 118
Copy link

Copilot AI Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable override logic is duplicated - it's already applied in all_project_variables on line 102. This second application is redundant since all_project_variables already includes the overrides and is applied to packages on lines 109 and 115.

Suggested change
if variable_overrides:
package.variables.update(variable_overrides)

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

@izeigerman izeigerman Sep 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong! Below is the scenario the previous logic didn't handle:

For a variable with name package_var and the following setup:

  1. --var 'package_var: overridden_from_cli'
  2. In the project's dbt_project.yml:
vars:
  target_package:
    package_var: overridden_from_root_project
  1. In the package's dbt_project.yml:
vars
  package_var: default_value

The old logic returned overridden_from_root_project instead of overridden_from_cli.

return Project(context, profile, packages)

Expand Down
10 changes: 10 additions & 0 deletions tests/dbt/test_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,16 @@ def test_variables(assert_exp_eq, sushi_test_project):
assert sushi_test_project.packages["customers"].variables == expected_customer_variables


@pytest.mark.slow
def test_variables_override(init_and_plan_context: t.Callable):
context, _ = init_and_plan_context(
"tests/fixtures/dbt/sushi_test", config="test_config_with_var_override"
)
dbt_project = context._loaders[0]._load_projects()[0] # type: ignore
assert dbt_project.packages["sushi"].variables["some_var"] == "overridden_from_config_py"
assert dbt_project.packages["customers"].variables["some_var"] == "overridden_from_config_py"


@pytest.mark.slow
def test_jinja_in_dbt_variables(sushi_test_dbt_context: Context):
assert sushi_test_dbt_context.render("sushi.top_waiters").sql().endswith("LIMIT 10")
Expand Down
10 changes: 10 additions & 0 deletions tests/fixtures/dbt/sushi_test/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,16 @@

test_config = config


test_config_with_var_override = sqlmesh_config(
Path(__file__).parent,
model_defaults=ModelDefaultsConfig(dialect="duckdb", start="Jan 1 2022"),
variables={
"some_var": "overridden_from_config_py",
},
)


test_config_with_normalization_strategy = sqlmesh_config(
Path(__file__).parent,
model_defaults=ModelDefaultsConfig(
Expand Down