Feat(sqlmesh_dbt): Implement --model and --resource-type#5443
Conversation
|
|
||
|
|
||
| def to_sqlmesh(dbt_select: t.Collection[str], dbt_exclude: t.Collection[str]) -> t.Optional[str]: | ||
| def consolidate( |
There was a problem hiding this comment.
Why even expose this outside this module? Why not make it a part of a single unified to_sqlmesh interface?
There was a problem hiding this comment.
I thought about that but I wanted to keep the surface of to_sqlmesh() simple and take the minimum required parameters.
combine() still operates in dbt-land and produces consolidated selectors in dbt format. The goal is to not have to always pass down select, exclude, resource_type, models etc parameters through every layer when really all of these parameters can be described in terms of select/exclude.
So they get consolidated early to reduce the number of parameters being passed around and then the lower layers then just call to_sqlmesh() on just select / exclude and dont have to worry about or consider any of the other parameters.
You can see this in DbtOperations._plan_builder which deliberately only needs select/exclude, not all the other options that may be passed.
| "exposure", | ||
| "snapshot", | ||
| "seed", | ||
| "default", |
There was a problem hiding this comment.
small question on the last two, what will all and default be in the sqlmesh context? I suppose this is a question for later on when multiple resource types support is added, but will these two mirror dbt as well?
There was a problem hiding this comment.
Currently, unimplemented :) but at a guess i'd say they would be aliases for the same thing because SQLMesh includes everything by default
This PR adds the
-m/--model/--modelsargument to thesqlmesh_dbtCLI.It does this by implementing support for the
resource_typeselector method in the SQLMesh selector engine and then if--modelsis supplied, injectingresource_type:modelinto the selector expression alongside the user-supplied values.Note that
--modelsis different to--select:--modelsjust includes model nodes - so excludes seeds, sources and tests--selectincludes all nodes