Feat(dbt): Add dbt graph context variable support#5159
Conversation
58fb39b to
1edb1b2
Compare
1edb1b2 to
6e20d76
Compare
| return { | ||
| "exposures": { | ||
| k: serializable(v.to_dict(omit_none=False)) | ||
| for k, v in getattr(self._manifest, "exposures", {}).items() |
There was a problem hiding this comment.
FMI: Is this a shortcut (along with serializable()) to just reach into the Manifest object returned by the dbt core lib and pull out fields, vs wrapping them in *Config objects and exposing them like we have done for self.models, self.seeds etc?
If so, this will probably need to be refactored in future if we want to do something else with this info
There was a problem hiding this comment.
I was thinking of this originally as we already convert the node objects for example into Config ones (models, seeds, tests, hooks etc.) but we then add extra sqlmesh-specific fields and map dbt terms to native ones in them. If we used those objects to construct the flat graph, we’d need to strip out the SQLMesh-specific attributes or the result wouldn't match what dbt returns. So since this is a user facing dictionary to be used in macros and not something we internally use I thought it better not to overcomplicate it and to construct the flat graph similar to dbt and use the already available dictionaries. That way when a user accesses it in a macro they get exactly what they would in dbt
There was a problem hiding this comment.
Yeah that makes sense.
I dont think we will ever get rid of the DBT manifest and the dependency on dbt-core :)
sqlmesh/utils/conversions.py
Outdated
| return val | ||
|
|
||
|
|
||
| def serializable(obj: t.Any) -> t.Any: |
There was a problem hiding this comment.
Let's use a verb so it's clear it serializes? Right now it reads like a property check, i.e. "is this serializable?"
6e20d76 to
91d7fa8
Compare
91d7fa8 to
d577edd
Compare
This adds support for the dbt
graphcontext variable which contains information about the nodes in a dbt project.Docs: https://docs.getdbt.com/reference/dbt-jinja-functions/graph