Skip to content

Feat(dbt): Add dbt graph context variable support#5159

Merged
themisvaltinos merged 3 commits intomainfrom
themis/dbtgraph
Aug 19, 2025
Merged

Feat(dbt): Add dbt graph context variable support#5159
themisvaltinos merged 3 commits intomainfrom
themis/dbtgraph

Conversation

@themisvaltinos
Copy link
Contributor

@themisvaltinos themisvaltinos commented Aug 14, 2025

This adds support for the dbt graph context variable which contains information about the nodes in a dbt project.

Docs: https://docs.getdbt.com/reference/dbt-jinja-functions/graph

@themisvaltinos themisvaltinos requested a review from a team August 14, 2025 20:08
return {
"exposures": {
k: serializable(v.to_dict(omit_none=False))
for k, v in getattr(self._manifest, "exposures", {}).items()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FMI: Is this a shortcut (along with serializable()) to just reach into the Manifest object returned by the dbt core lib and pull out fields, vs wrapping them in *Config objects and exposing them like we have done for self.models, self.seeds etc?

If so, this will probably need to be refactored in future if we want to do something else with this info

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of this originally as we already convert the node objects for example into Config ones (models, seeds, tests, hooks etc.) but we then add extra sqlmesh-specific fields and map dbt terms to native ones in them. If we used those objects to construct the flat graph, we’d need to strip out the SQLMesh-specific attributes or the result wouldn't match what dbt returns. So since this is a user facing dictionary to be used in macros and not something we internally use I thought it better not to overcomplicate it and to construct the flat graph similar to dbt and use the already available dictionaries. That way when a user accesses it in a macro they get exactly what they would in dbt

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that makes sense.

I dont think we will ever get rid of the DBT manifest and the dependency on dbt-core :)

return val


def serializable(obj: t.Any) -> t.Any:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use a verb so it's clear it serializes? Right now it reads like a property check, i.e. "is this serializable?"

@themisvaltinos themisvaltinos enabled auto-merge (squash) August 19, 2025 02:22
@themisvaltinos themisvaltinos merged commit 98c858c into main Aug 19, 2025
27 of 28 checks passed
@themisvaltinos themisvaltinos deleted the themis/dbtgraph branch August 19, 2025 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants