diff --git a/docs.json b/docs.json index 2cb4d473..5fe15638 100644 --- a/docs.json +++ b/docs.json @@ -86,6 +86,13 @@ "guides/how-to-promote-content", "guides/keyboard-shortcuts", "guides/table-calculations", + { + "group": "Table calculation functions", + "pages": [ + "references/table-calculation-functions/row-functions", + "references/table-calculation-functions/pivot-functions" + ] + }, { "group": "Table calc SQL templates", "pages": [ diff --git a/guides/table-calculations.mdx b/guides/table-calculations.mdx index d154f96c..8e2824a2 100644 --- a/guides/table-calculations.mdx +++ b/guides/table-calculations.mdx @@ -53,7 +53,7 @@ Once you've got some data in your results table, you can create a table calculat ##### Write the SQL for your table calculation in the pop-up box -Your table calculation is defined using raw SQL that you write up in this pop up box. If you're not sure what to write here, you can [check out some of our table calculation SQL templates](#table-calculation-sql-templates). +Your table calculation is defined using raw SQL that you write up in this pop up box. If you're not sure what to write here, you can [check out some of our table calculation SQL templates](#table-calculation-functions-and-sql-templates). To reference the metrics and dimensions in your results table, you can either use the autocomplete, or you can manually write the full field name using the format `${table_name.field_name}`. @@ -62,6 +62,15 @@ To reference the metrics and dimensions in your results table, you can either us +### Using table calculation functions + +In addition to writing raw SQL, you can use built-in functions that simplify common table calculation patterns: + +- **[Row functions](/references/table-calculation-functions/row-functions)** — Access values from other rows (e.g., previous row, specific row number, lookups) +- **[Pivot functions](/references/table-calculation-functions/pivot-functions)** — Access values across pivot columns + +These functions compile to SQL window functions automatically, so you don't need to write `LAG`, `LEAD`, or `ROW_NUMBER` by hand. + ##### Update the format of your table calculation (if needed) If needed, you can update the format of your table calculation to things like percent formatting using the `format` tab. @@ -105,8 +114,8 @@ For example: You use table calculations to calculate the `percentage_of_shipping ![](/images/guides/table-calculations/table_calculation_filter_example.jpg) -## Table calculation SQL templates +## Table calculation functions and SQL templates -I'm not sure about you, but I definitely rely on copy-pasting old SQL code to write 90% of my new SQL queries. So, we thought it might be useful to give you some snippets of SQL code to help you get started with your most common table calculations. +If you prefer not to write raw SQL, check out the built-in [row functions](/references/table-calculation-functions/row-functions) and [pivot functions](/references/table-calculation-functions/pivot-functions) — they handle common patterns like accessing previous rows, row numbering, and working across pivot columns. -You can check out our table calculation SQL templates [here](/guides/table-calculations/sql-templates). +For more advanced or custom SQL, we also have copy-paste SQL templates to help you get started with common table calculations. You can check out our table calculation SQL templates [here](/guides/table-calculations/sql-templates). diff --git a/guides/table-calculations/sql-templates.mdx b/guides/table-calculations/sql-templates.mdx index a52de417..6ab97f1e 100644 --- a/guides/table-calculations/sql-templates.mdx +++ b/guides/table-calculations/sql-templates.mdx @@ -5,5 +5,9 @@ mode: wide sidebarTitle: Overview --- + +Looking for a simpler alternative to raw SQL? Check out the built-in [row functions](/references/table-calculation-functions/row-functions) and [pivot functions](/references/table-calculation-functions/pivot-functions) — they handle common patterns like accessing previous rows and working across pivot columns without needing to write window function SQL. + + diff --git a/references/table-calculation-functions/pivot-functions.mdx b/references/table-calculation-functions/pivot-functions.mdx new file mode 100644 index 00000000..85314d44 --- /dev/null +++ b/references/table-calculation-functions/pivot-functions.mdx @@ -0,0 +1,98 @@ +--- +title: "Pivot functions" +description: "Built-in functions for accessing values across pivot columns in your table calculations." +--- + + + **Table calculation functions are an Experimental feature.** + + These functions may change or be updated without notice as we iterate. See [feature maturity levels](/references/workspace/feature-maturity-levels) for details. + + +Pivot functions let you work with values across pivot columns in your results table. When you pivot a dimension in Lightdash, the values of that dimension become separate columns — pivot functions give you a way to reference and aggregate across those columns. + + +Pivot functions are only available when your query includes a pivoted dimension. + + +## pivot_row + +Returns an array of all values across the pivot columns for the current row. + +``` +pivot_row(expression) +``` + +| Parameter | Type | Description | +| :--- | :--- | :--- | +| `expression` | column reference or SQL expression | The expression to evaluate for each pivot column | + +**Example** + +Get all pivoted revenue values for the current row: + +``` +pivot_row(${orders.total_revenue}) +``` + + +```sql +ARRAY_AGG(${orders.total_revenue}) OVER ( + PARTITION BY "row_index" + ORDER BY "column_index" + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING +) +``` + + +--- + +## pivot_offset_list + +Returns an array of values from consecutive pivot columns starting at a relative offset from the current column. + +``` +pivot_offset_list(expression, columnOffset, numValues) +``` + +| Parameter | Type | Description | +| :--- | :--- | :--- | +| `expression` | column reference or SQL expression | The expression to evaluate | +| `columnOffset` | integer | Starting column offset. Negative = previous columns, positive = next columns, 0 = current column | +| `numValues` | integer | Number of consecutive pivot columns to include | + +Values are returned as `NULL` when the offset points to a non-adjacent pivot column (e.g., if columns were filtered out). + +**Example** + +Get the current and two previous pivot column values: + +``` +pivot_offset_list(${orders.total_revenue}, -2, 3) +``` + + +```sql +ARRAY[ + CASE WHEN LAG("column_index", 2) OVER ( + PARTITION BY "row_index" ORDER BY "column_index" + ) = "column_index" + (-2) + THEN LAG(${orders.total_revenue}, 2) OVER ( + PARTITION BY "row_index" ORDER BY "column_index" + ) + ELSE NULL + END, + CASE WHEN LAG("column_index", 1) OVER ( + PARTITION BY "row_index" ORDER BY "column_index" + ) = "column_index" + (-1) + THEN LAG(${orders.total_revenue}, 1) OVER ( + PARTITION BY "row_index" ORDER BY "column_index" + ) + ELSE NULL + END, + ${orders.total_revenue} +] +``` + +Each element includes an adjacency guard — a `CASE WHEN` check that verifies the target column is actually adjacent. This prevents incorrect values when pivot columns have been filtered out and are non-contiguous. + diff --git a/references/table-calculation-functions/row-functions.mdx b/references/table-calculation-functions/row-functions.mdx new file mode 100644 index 00000000..ac461285 --- /dev/null +++ b/references/table-calculation-functions/row-functions.mdx @@ -0,0 +1,214 @@ +--- +title: "Row functions" +description: "Built-in functions for accessing values from other rows in your table calculations." +--- + + + **Table calculation functions are an Experimental feature.** + + These functions may change or be updated without notice as we iterate. See [feature maturity levels](/references/workspace/feature-maturity-levels) for details. + + +Row functions let you reference values from other rows without writing raw SQL window functions. They use the `${table.column}` syntax to reference fields in your results table. + + +Row functions use your query's current sort order to determine row positions. Changing the sort order of your results will change the output of these functions. + + +## row + +Returns the 1-based row number of the current row. + +``` +row() +``` + +**Parameters:** None + +**Example** + +Add a row number column to your results: + +``` +row() +``` + + +```sql +ROW_NUMBER() OVER (ORDER BY ...) +``` + +The `ORDER BY` clause uses the sort order configured in your query. + + +--- + +## offset + +Returns the value of a column from a row at a relative offset from the current row. + +``` +offset(column, rowOffset) +``` + +| Parameter | Type | Description | +| :--- | :--- | :--- | +| `column` | column reference | The column to get the value from | +| `rowOffset` | integer | Number of rows to offset. Negative = previous rows, positive = next rows, 0 = current row | + +**Example** + +Get the previous row's revenue to calculate period-over-period changes: + +``` +${orders.total_revenue} - offset(${orders.total_revenue}, -1) +``` + + +For negative offsets (previous rows): +```sql +LAG(${orders.total_revenue}, 1) OVER (ORDER BY ...) +``` + +For positive offsets (next rows): +```sql +LEAD(${orders.total_revenue}, 1) OVER (ORDER BY ...) +``` + +For an offset of 0, the column value is returned directly with no window function. + + +--- + +## index + +Returns the value of an expression from an absolute row position (1-based). + +``` +index(expression, rowIndex) +``` + +| Parameter | Type | Description | +| :--- | :--- | :--- | +| `expression` | column reference or SQL expression | The expression to evaluate | +| `rowIndex` | integer (≥ 1) | The 1-based row position to get the value from | + +**Example** + +Compare every row's revenue against the first row's revenue: + +``` +${orders.total_revenue} / index(${orders.total_revenue}, 1) +``` + + +```sql +NTH_VALUE(${orders.total_revenue}, 1) OVER ( + ORDER BY ... + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING +) +``` + + +--- + +## lookup + +Finds a value in one column and returns the corresponding value from another column. + +``` +lookup(value, lookupColumn, resultColumn) +``` + +| Parameter | Type | Description | +| :--- | :--- | :--- | +| `value` | any | The value to search for | +| `lookupColumn` | column reference | The column to search in | +| `resultColumn` | column reference | The column to return the value from | + +If multiple rows match, the largest matching value from `resultColumn` is returned. + +**Example** + +Look up the revenue for a specific status: + +``` +lookup('completed', ${orders.status}, ${orders.total_revenue}) +``` + + +```sql +MAX( + CASE WHEN ${orders.status} = 'completed' + THEN ${orders.total_revenue} + ELSE NULL + END +) OVER () +``` + + +--- + +## offset_list + +Returns an array of values from consecutive rows starting at a relative offset. + +``` +offset_list(column, rowOffset, numValues) +``` + +| Parameter | Type | Description | +| :--- | :--- | :--- | +| `column` | column reference | The column to get values from | +| `rowOffset` | integer | Starting row offset. Negative = previous rows, positive = next rows, 0 = current row | +| `numValues` | integer | Number of consecutive values to include | + +**Example** + +Get the current and two previous revenue values (a 3-period window): + +``` +offset_list(${orders.total_revenue}, -2, 3) +``` + + +```sql +ARRAY[ + LAG(${orders.total_revenue}, 2) OVER (ORDER BY ...), + LAG(${orders.total_revenue}, 1) OVER (ORDER BY ...), + ${orders.total_revenue} +] +``` + + +--- + +## list + +Constructs an array from multiple values. Unlike the other row functions, `list` does not use any windowing. + +``` +list(value1, value2, ..., valueN) +``` + +| Parameter | Type | Description | +| :--- | :--- | :--- | +| `value1...valueN` | any | Values or expressions to combine into an array | + +**Example** + +Create an array of specific column values: + +``` +list(${orders.total_revenue}, ${orders.total_cost}, ${orders.total_profit}) +``` + + +```sql +ARRAY[ + ${orders.total_revenue}, + ${orders.total_cost}, + ${orders.total_profit} +] +``` +