You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/concepts/audits.md
+26Lines changed: 26 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,6 +9,32 @@ A comprehensive suite of audits can identify data issues upstream, whether they
9
9
10
10
**NOTE**: For incremental by time range models, audits are only applied to intervals being processed - not for the entire underlying table.
11
11
12
+
## Blocking audits
13
+
A failed blocking audit halts the execution of a `plan` or `run` to prevent invalid data from propagating to downstream models. The impact of a failure depends on whether you are running a `plan` or a `run`.
14
+
15
+
SQLMesh's blocking audit process is:
16
+
17
+
1. Evaluate the model (e.g., insert new data or rebuild the table)
18
+
2. Run the audit query against the newly updated model table. For incremental models, the audit only runs on the processed time intervals.
19
+
3. If the query returns any rows, the audit fails, halting the `plan` or `run`.
20
+
21
+
### Plan vs. Run
22
+
23
+
The key difference is when the model's data is promoted to the production environment:
24
+
25
+
***`plan`**: SQLMesh evaluates and audits all modified models *before* promoting them to production. If an audit fails, the `plan` stops, and the production table is untouched. Invalid data is contained in an isolated table and never reaches the production environment.
26
+
27
+
***`run`**: SQLMesh evaluates and audits models directly against the production environment. If an audit fails, the `run` stops, but the invalid data *is already present* in the production table. The "blocking" action prevents this bad data from being used to build other downstream models.
28
+
29
+
### Fixing a Failed Audit
30
+
31
+
If a blocking audit fails during a `run`, you must fix the invalid data in the production table. To do so:
32
+
33
+
1.**Find the root cause**: examine upstream models and data sources
34
+
2.**Fix the source**
35
+
* If the cause is an **external data source**, fix it there. Then, run a [restatement plan](./plans.md#restatement-plans) on the first SQLMesh model that ingests the source data. This will restate all downstream models, including the one with the failed audit.
36
+
* If the cause is a **SQLMesh model**, update the model's logic. Then apply the change with a `plan`, which will automatically re-evaluate all downstream models.
37
+
12
38
## User-Defined Audits
13
39
In SQLMesh, user-defined audits are defined in `.sql` files in an `audits` directory in your SQLMesh project. Multiple audits can be defined in a single file, so you can organize them to your liking. Alternatively, audits can be defined inline within the model definition itself.
0 commit comments