Skip to content

Feat: dbt cli skeleton#5118

Merged
erindru merged 2 commits intomainfrom
erin/dbt-cli
Aug 11, 2025
Merged

Feat: dbt cli skeleton#5118
erindru merged 2 commits intomainfrom
erin/dbt-cli

Conversation

@erindru
Copy link
Collaborator

@erindru erindru commented Aug 11, 2025

Adds a very basic skeleton for creating a CLI that takes input in the form of existing dbt core CLI commands and translates them to the SQLMesh equivalent.

It doesn't do much yet but i've raised this PR to demonstrate the approach i'm thinking and also in the spirit of smaller, more focused PR's vs a big dump.

The goal here is also to provide some initial hooks to help multiple developers build out functionality in parallel.

Key points:

  • New CLI entrypoint, sqlmesh_dbt, with the intention of being able to one day alias dbt=sqlmesh_dbt and still have things work
  • It's currently in its own top-level package to avoid the delay that occurs when importing anything from the sqlmesh package. This allows it to start outputting to the console immediately and defer loading SQLMesh until later.
  • I added a basic test skeleton that is based around the public jaffle_shop_duckdb project
  • I also updated ruff/mypy/setuptools etc to pick up the new top-level package

If you clone the jaffle shop_duckdb project, you can, with zero changes to anything in that project:

$ sqlmesh_dbt list
Loading engine ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  
[WARNING] DuckDB does not support concurrency - setting threads to 1.
05:11:20  Registered adapter: duckdb=1.9.2
main.orders
main.customers
main.stg_payments
...SNIP..

(note: I only added this because it was helpful in investigating why selectors weren't matching)

and

$ sqlmesh_dbt run
Loading project ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 
05:13:03  Registered adapter: duckdb=1.9.2

`prod` environment will be initialized

..SNIP... usual SQLMesh output for a forward-only no-prompts plan

select_models=select_models,
forward_only=True,
no_auto_categorization=True, # everything is breaking / foward-only
effective_from=self.context.config.model_defaults.start,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

effective_from means something different, let's use start instead or not pass anything at all


self.context.plan(
select_models=select_models,
forward_only=True,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should not set this explicitly

progress.update(load_task_id, description="Loading project", total=None)

# inject default start date if one is not specified to prevent the user from having to do anything
_inject_default_start_date(project_dir)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing this I'd suggest extending sqlmesh_config with the start date and hardcode the start date at sqlmesh init time.

Copy link
Collaborator Author

@erindru erindru Aug 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do this in a follow-up PR. To summarize our conversation:

  • We dont touch any existing dbt file like dbt_project.yml
  • We check for a file called sqlmesh.yaml in the root of the DBT project
  • If it exists, we read directives like the start date from it and it generally follows the same format as SQLMesh's existing config.yaml file
  • If it doesn't exist, we do an "automatic init" to create it and stamp the start date as yesterday_ds() so it can take effect on subsequent invocations

This approach also provides a place to store other configuration in future, such as the virtual data environment mode

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -0,0 +1 @@
pytestmark = ["foo"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's leftover from when I was experimenting with pytest marks. Will remove it

@erindru erindru marked this pull request as ready for review August 11, 2025 22:41
@erindru erindru merged commit 1237e9a into main Aug 11, 2025
27 of 28 checks passed
@erindru erindru deleted the erin/dbt-cli branch August 11, 2025 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants