Skip to content

Commit a1feec9

Browse files
mesmith027treysp
andauthored
[DOCS]: MotherDuck additions (#3870)
Co-authored-by: Trey Spiller <1831878+treysp@users.noreply.github.com>
1 parent a94b11f commit a1feec9

File tree

5 files changed

+90
-0
lines changed

5 files changed

+90
-0
lines changed

docs/integrations/engines/motherduck.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,96 @@
11
# MotherDuck
22

3+
This page provides information about how to use SQLMesh with MotherDuck.
4+
5+
It begins with a [Connection Quickstart](#connection-quickstart) that demonstrates how to connect to MotherDuck, or you can skip directly to information about using MotherDuck with the built-in or airflow schedules.
6+
7+
## Connection quickstart
8+
9+
Connecting to cloud warehouses involves a few steps, so this connection quickstart provides the info you need to get up and running with MotherDuck.
10+
11+
It demonstrates connecting to MotherDuck with the `duckdb` library bundled with SQLMesh.
12+
13+
MotherDuck provides a single way to authorize a connection. This quickstart demonstrates authenticating with a token.
14+
15+
!!! tip
16+
This quick start assumes you are familiar with basic SQLMesh commands and functionality.
17+
18+
If you’re not familiar, work through the [SQLMesh Quickstart](../../quick_start.md) before continuing.
19+
20+
### Prerequisites
21+
22+
Before working through this quickstart guide, ensure that:
23+
24+
1. You have a motherduck account and an access token.
25+
2. Your computer has SQLMesh installed with the DuckDB extra available.
26+
1. Install from command line with the command `pip install “sqlmesh[duckdb]”`
27+
3. You have initialized a SQLMesh example project on your computer
28+
1. Open a command line interface and navigate to the directory where the project files should go.
29+
2. Initialize the project with the command `sqlmesh init motherduck`
30+
31+
#### Access control permissions
32+
33+
SQLMesh must have sufficient permissions to create and access your MotherDuck databases. Since permission is granted to specific databases for a specific user, you should create a service account for SQLMesh that will contain the credentials for writing to MotherDuck.
34+
35+
### Configure the connection
36+
37+
We now have what is required to configure SQLMesh’s connection to MotherDuck.
38+
39+
We start the configuration by adding a gateway named `motherduck` to our example project’s config.yaml file and making it our `default gateway`, as well as adding our token, persistent, and ephemeral catalogs.
40+
41+
```yaml
42+
gateways:
43+
motherduck:
44+
connection:
45+
type: motherduck
46+
catalogs:
47+
persistent: ‘md:’
48+
ephemeral: ‘:memory:’
49+
token: \<your\_token\>
50+
51+
default\_gateway: motherduck
52+
```
53+
54+
!!! warning
55+
Best practice for storing secrets like tokens is placing them in [environment variables that the configuration file loads dynamically](../../guides/configuration.md#environment-variables). For simplicity, this guide instead places the value directly in the configuration file.
56+
57+
This code demonstrates how to use the environment variable `MOTHERDUCK_TOKEN` for the configuration's `token` parameter:
58+
59+
```yaml linenums="1" hl_lines="5"
60+
gateways:
61+
motherduck:
62+
connection:
63+
type: motherduck
64+
token: {{ env_var('MOTHERDUCK_TOKEN') }}
65+
```
66+
67+
### Check connection
68+
69+
We have now specified the `motherduck` gateway connection information, so we can confirm that SQLMesh is able to successfully connect to MotherDuck. We will test the connection with the `sqlmesh info` command.
70+
71+
First, open a command line terminal. Now enter the command `sqlmesh info`:
72+
73+
![](./motherduck/sqlmesh_info.png)
74+
75+
The output shows that our data warehouse connection succeeded:
76+
77+
![](./motherduck/info_output.png)
78+
79+
### Run a `sqlmesh plan`
80+
81+
Now we're ready to run a `sqlmesh plan` in MotherDuck:
82+
83+
![](./motherduck/sqlmesh_plan.png)
84+
85+
And confirm that our schemas and objects exist in the MotherDuck catalog:
86+
87+
![](./motherduck/motherduck_ui.png)
88+
89+
Congratulations \- your SQLMesh project is up and running on MotherDuck\!
90+
91+
392
## Local/Built-in Scheduler
93+
494
**Engine Adapter Type**: `motherduck`
595

696
### Connection options
82.4 KB
Loading
172 KB
Loading
65.1 KB
Loading
136 KB
Loading

0 commit comments

Comments
 (0)