i’m using sqlmesh with a python model that reads data from pickled dataframes and inserts into a slowly changing dimension (scd type 2) table on mssql server.
i’ve configured the model with:
@model(
...
physical_properties={"mssql_merge_exists": True}
)
however, when the query is rendered in the logs, there is no usage of the MERGE keyword. instead, the generated sql falls back to a large series of WITH ... ctes, unions, and inserts.
this behavior is causing two issues:
-
the model does not run successfully — it crashes after ~20 minutes of execution.
-
sql server resource usage spikes heavily (likely because of the expanded cte/unions strategy instead of a merge).