Skip to content

Commit 911bcfc

Browse files
committed
docs feedback
1 parent bfa6af6 commit 911bcfc

File tree

2 files changed

+18
-3
lines changed

2 files changed

+18
-3
lines changed

docs/guides/configuration.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -446,8 +446,9 @@ Out of the box, SQLMesh has the following defaults set:
446446

447447
- `environment_suffix_target: schema`
448448
- `physical_table_naming_convention: schema_and_table`
449+
- no `physical_schema_mapping` overrides, so a `sqlmesh__<model schema>` physical schema will be created for each model schema
449450

450-
Given a catalog of `warehouse` and a model named `finance_mart.transaction_events_over_threshold`, this causes SQLMesh to create physical tables using the following convention:
451+
This means that given a catalog of `warehouse` and a model named `finance_mart.transaction_events_over_threshold`, SQLMesh will create physical tables using the following convention:
451452

452453
```
453454
# <catalog>.sqlmesh__<schema>.<schema>__<table>__<fingerprint>
@@ -457,6 +458,8 @@ warehouse.sqlmesh__finance_mart.finance_mart__transaction_events_over_threshold_
457458

458459
This deliberately contains some redundancy with the *model* schema as it's repeated at the physical layer in both the physical schema name as well as the physical table name.
459460

461+
This default exists to make the physical table names portable between different configurations. If you were to define a `physical_schema_mapping` that maps all models to the same physical schema, since the model schema is included in the table name as well, there are no naming conflicts.
462+
460463
##### Table only
461464

462465
Some engines have object name length limitations which cause them to [silently truncate](https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS) table and view names that exceed this limit. This behaviour breaks SQLMesh, so we raise a runtime error if we detect the engine would silently truncate the name of the table we are trying to create.
@@ -489,6 +492,18 @@ warehouse.sqlmesh__finance_mart.transaction_events_over_threshold__<fingerprint>
489492

490493
Notice that the model schema name is no longer part of the physical table name. This allows for slightly longer model names on engines with low identifier length limits, which may be useful for your project.
491494

495+
In this configuration, it is your responsibility to ensure that any schema overrides in `physical_schema_mapping` result in each model schema getting mapped to a unique physical schema.
496+
497+
For example, the following configuration will cause **data corruption**:
498+
499+
```yaml
500+
physical_table_naming_convention: table_only
501+
physical_schema_mapping:
502+
'.*': sqlmesh
503+
```
504+
505+
This is because every model schema is mapped to the same physical schema but the model schema name is omitted from the physical table name.
506+
492507
##### MD5 hash
493508

494509
If you *still* need more characters, you can set `physical_table_naming_convention: hash_md5` like so:
@@ -521,7 +536,7 @@ sqlmesh_md5__d3b07384d113edec49eaa6238ad5ff00
521536
sqlmesh_md5__d3b07384d113edec49eaa6238ad5ff00__dev
522537
```
523538

524-
This has a downside that now it's much more difficult to determine which table corresponds to which model by just looking at the database with a SQL client. However, the table names now have a predictable length so there are no longer any surprises with identfiers exceeding the max length at the physical layer.
539+
This has a downside that now it's much more difficult to determine which table corresponds to which model by just looking at the database with a SQL client. However, the table names have a predictable length so there are no longer any surprises with identfiers exceeding the max length at the physical layer.
525540

526541
#### Environment view catalogs
527542

tests/core/test_integration.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7117,7 +7117,7 @@ def test_engine_adapters_multi_repo_all_gateways_gathered(copy_to_temp_path):
71177117
expected_gateways = {"local", "memory", "extra"}
71187118
assert gathered_gateways == expected_gateways
71197119

7120-
7120+
71217121
def test_physical_table_naming_strategy_table_only(copy_to_temp_path: t.Callable):
71227122
sushi_context = Context(
71237123
paths=copy_to_temp_path("examples/sushi"),

0 commit comments

Comments
 (0)