-
Notifications
You must be signed in to change notification settings - Fork 5
feat: document the blueprint sources #130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| import { Meta } from "nextra"; | ||
|
|
||
| const meta: Meta = { | ||
| introduction: "Introduction", | ||
| sources: "Sources", | ||
| }; | ||
|
|
||
| export default meta; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # Deploying Blueprints | ||
|
|
||
| In order for a blueprint to be instance-able, it'll need to be deployed and have one or more sources defined. | ||
|
|
||
| ## Defining Sources | ||
|
|
||
| Before deploying, you must specify the sources from which to fetch the blueprint binaries. | ||
| See [Sources](/developers/deployment/sources/introduction). | ||
|
|
||
| ## Deploying via the CLI | ||
|
|
||
| Once you're ready to deploy your blueprint, simply: | ||
|
|
||
| Install the [cargo-tangle] CLI, if you haven't already. | ||
|
|
||
| For testing, you can deploy to a local devnet: | ||
|
|
||
| ```shell | ||
| export SIGNER="//Alice" | ||
| export EVM_SIGNER="0xcb6df9de1efca7a3998a8ead4e02159d5fa99c3e0d4fd6432667390bb4726854" | ||
| cargo tangle blueprint deploy tangle --devnet | ||
| ``` | ||
|
|
||
| See [deploy command reference](/developers/cli/tangle#deploying-a-blueprint) for all options. | ||
|
|
||
| [cargo-tangle]: /developers/cli/installation |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| import { Meta } from "nextra"; | ||
|
|
||
| const meta: Meta = { | ||
| introduction: "Introduction", | ||
| native: "Native", | ||
| container: "Container", | ||
| tee: "Trusted Execution Environment (WIP)", | ||
| wasm: "WASM (WIP)", | ||
| testing: "Testing", | ||
| }; | ||
|
|
||
| export default meta; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # Container Source | ||
|
|
||
| The `Container` source is used for blueprints that are intended to be built as container images and published to image | ||
| registries, such as `docker.io`. | ||
|
|
||
| ## Requirements | ||
|
|
||
| The requirements for running containerized blueprints are available [here](/operators/manager/requirements#container-sources) | ||
|
|
||
| ## Format | ||
|
|
||
| The `Container` source has the following format: | ||
|
|
||
| ```rust | ||
| pub struct ContainerSource { | ||
| pub registry: String, | ||
| pub image: String, | ||
| pub tag: String, | ||
| } | ||
| ``` | ||
|
|
||
| Where: | ||
|
|
||
| - `registry` - The registry to pull the image from (e.g., `docker.io`) | ||
| - `image` - The name of the image (e.g., `some-user/my-blueprint`) | ||
| - `tag` - The release tag of the image (e.g., `latest`) | ||
|
|
||
| And they can be specified in the manifest of your binary crate like so: | ||
|
|
||
| ```toml | ||
| [package.metadata.blueprint] | ||
| sources = [ | ||
| { type = "Container", registry = "docker.io", image = "some-user/my-blueprint", tag = "latest" } | ||
| ] | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Blueprint Sources | ||
|
|
||
| Blueprints can be built and distributed in multiple formats (visible on the sidebar). | ||
|
|
||
| A blueprint can define many sources of different types in the manifest of its binary crate: | ||
|
|
||
| ```toml | ||
| [package.metadata.blueprint] | ||
| sources = [ | ||
| { type = "Container", registry = "docker.io", image = "some-user/my-blueprint", tag = "latest" }, | ||
| # Fallback container source | ||
| { type = "Container", registry = "ghcr.io", image = "some-user/my-blueprint", tag = "latest" }, | ||
| # Native binary source | ||
| { type = "Native", owner = "some-github-user", repo = "some-blueprint", tag = "0.1.0", binaries = [ | ||
| { arch = "Amd64", os = "Linux", name = "my-blueprint-bin" }, | ||
| ] }, | ||
| ] | ||
| ``` | ||
|
|
||
| The above example has two container sources, which allows an operator to attempt a pull from different sources in the event | ||
| that one of the registries is unreachable. Additionally, it has a native source which can act as a fallback for operators | ||
| that either cannot or prefer not to support running containerized blueprints. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Native Sources | ||
|
|
||
| The `Native` source is used for blueprints that compile to a native binary and release via the [`release.yml`] GitHub workflow | ||
| from the blueprint template. | ||
|
|
||
| ## Requirements | ||
|
|
||
| The requirements for running native blueprints are available [here](/operators/manager/requirements#native-sources) | ||
|
|
||
| ## Format | ||
|
|
||
| The `Native` source has the following format: | ||
|
|
||
| ```rust | ||
| pub struct NativeSource { | ||
| pub owner: String, | ||
| pub repo: String, | ||
| pub tag: String, | ||
| pub binaries: Vec<BlueprintBinary>, | ||
| } | ||
|
|
||
| pub struct BlueprintBinary { | ||
| pub arch: Architecture, | ||
| pub os: OperatingSystem, | ||
| pub name: String, | ||
| } | ||
| ``` | ||
|
|
||
| Where: | ||
|
|
||
| - `owner` - The owner of the GitHub repository (e.g., `tangle-network`) | ||
| - `repo` - The name of the repository (e.g., `some-blueprint`) | ||
| - `tag` - The release tag of the binary (e.g., `0.1.0`) | ||
| - `binaries` - A list of binaries (e.g., see below) | ||
|
|
||
| And in `BlueprintBinary`: | ||
|
|
||
| - `arch` - The architecture the blueprint was built for (see [Architecture]) | ||
| - `os` - The operating system the blueprint was built for (see [OperatingSystem]) | ||
| - `name` - The name of the binary in the GitHub release (e.g., `my-blueprint-bin`) | ||
|
|
||
| And they can be specified in the manifest of your binary crate like so: | ||
|
|
||
| ```toml | ||
| [package.metadata.blueprint] | ||
| sources = [ | ||
| { type = "Native", owner = "some-github-user", repo = "some-blueprint", tag = "0.1.0", binaries = [ | ||
| { arch = "Amd64", os = "Linux", name = "my-blueprint-bin" }, | ||
| ] }, | ||
| ] | ||
| ``` | ||
|
|
||
| [`release.yml`]: https://github.com/tangle-network/blueprint-template/blob/main/.github/workflows/release.yml | ||
| [Architecture]: https://docs.rs/tangle-subxt/latest/tangle_subxt/tangle_testnet_runtime/api/runtime_types/tangle_primitives/services/sources/enum.Architecture.html | ||
| [OperatingSystem]: https://docs.rs/tangle-subxt/latest/tangle_subxt/tangle_testnet_runtime/api/runtime_types/tangle_primitives/services/sources/enum.OperatingSystem.html |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # Trusted Execution Environment Sources | ||
|
|
||
| The `TEE` (Trusted Execution Environment) source is used for blueprints that are built as container images, and intended to | ||
| be deployed to a [dstack] TEE. | ||
|
|
||
| ## Requirements | ||
|
|
||
| The requirements for running TEE blueprints are available [here](/operators/manager/requirements#tee-sources) | ||
|
|
||
| ## Format | ||
|
|
||
| The `TEE` source has the following format: | ||
|
|
||
| ```rust | ||
| // TBD... | ||
| ``` | ||
|
|
||
| Where: | ||
|
|
||
| - TODO | ||
|
|
||
| And they can be specified in the manifest of your binary crate like so: | ||
|
|
||
| ```toml | ||
| [package.metadata.blueprint] | ||
| sources = [ | ||
| { type = "TEE", ... }, | ||
| ] | ||
| ``` | ||
|
|
||
| [dstack]: https://github.com/Dstack-TEE/dstack |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| # Testing Source | ||
|
|
||
| The `Testing` source is a special, automatically generated source that was historically used by the [Blueprint Manager] | ||
| during integration tests. It has since been replaced with the [test harnesses](/developers/testing/introduction). | ||
|
|
||
| ## Format | ||
|
|
||
| The `Testing` source has the following format: | ||
|
|
||
| ```rust | ||
| pub struct TestingSource { | ||
| pub cargo_package: String, | ||
| pub cargo_bin: String, | ||
| pub base_path: String, | ||
| } | ||
| ``` | ||
|
|
||
| Where: | ||
|
|
||
| - `cargo_package` - The Cargo package name of the binary (e.g., `my-blueprint-bin`) | ||
| - `cargo_bin` - The name of the built binary file (e.g., `my-blueprint-bin`) | ||
| - `base_path` - The base path of the binary crate (e.g., `/home/user/my-blueprint/bin`) | ||
|
|
||
| And they can be specified in the manifest of your binary crate like so: | ||
|
|
||
| ```toml | ||
| [package.metadata.blueprint] | ||
| sources = [ | ||
| { type = "Testing", cargo_package = "some-package-name", cargo_bin = "some-binary", base_path = "/home/user/some-blueprint/bin" } | ||
| ] | ||
| ``` | ||
|
|
||
| ## Output | ||
|
|
||
| After building your blueprint, you'll notice something like the following in your `blueprint.json`: | ||
|
|
||
| ```json | ||
| { | ||
| // ... | ||
| "sources": [ | ||
| { | ||
| "type": "Testing", | ||
| "cargo_package": "my-blueprint-bin", | ||
| "cargo_bin": "my-blueprint-bin", | ||
| "base_path": "/home/user/my-blueprint/bin" | ||
| } | ||
| ], | ||
| } | ||
| ``` | ||
|
|
||
| It can safely be ignored (or deleted if you prefer). | ||
|
|
||
| [Blueprint Manager]: /operators/manager/introduction | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| # WebAssembly Sources | ||
|
|
||
| The `WASM` (WebAssembly) source is used for blueprints that are built as WASM binaries and intended to be executed in | ||
| a WASM runtime such as [Wasmtime]. | ||
|
|
||
| ## Requirements | ||
|
|
||
| The requirements for running WASM blueprints are available [here](/operators/manager/requirements#wasm-sources) | ||
|
|
||
| ## Format | ||
|
|
||
| The `WASM` source has the following format: | ||
|
|
||
| ```rust | ||
| pub struct WasmSource { | ||
| pub runtime: WasmRuntime, | ||
| pub fetcher: NativeSource, | ||
| } | ||
| ``` | ||
|
|
||
| Where: | ||
|
|
||
| - `runtime` - The WASM runtime this blueprint is intended to execute in (see [WasmRuntime]) | ||
| - `fetcher` - The GitHub release information, identical to [NativeSource] | ||
|
|
||
| And they can be specified in the manifest of your binary crate like so: | ||
|
|
||
| ```toml | ||
| [package.metadata.blueprint] | ||
| sources = [ | ||
| { type = "Wasm", runtime = "Wasmtime", fetcher = { owner = "some-github-user", repo = "some-blueprint", tag = "0.1.0", binaries = [ | ||
| { arch = "Wasm", os = "Unknown", name = "my-blueprint-bin" }, | ||
| ] } }, | ||
| ] | ||
| ``` | ||
|
|
||
| [Wasmtime]: https://wasmtime.dev/ | ||
| [NativeSource]: /developers/deployment/sources/native#format | ||
| [WasmRuntime]: https://docs.rs/tangle-subxt/latest/tangle_subxt/tangle_testnet_runtime/api/runtime_types/tangle_primitives/services/sources/enum.WasmRuntime.html |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| import { Meta } from "nextra"; | ||
|
|
||
| const meta: Meta = { | ||
| introduction: "Introduction", | ||
| requirements: "Requirements", | ||
| }; | ||
|
|
||
| export default meta; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Blueprint Manager | ||
|
|
||
| TODO |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # Runtime Requirements for the Blueprint Manager | ||
|
|
||
| Blueprints can be executed in multiple ways (see [Sources](/developers/deployment/sources/introduction)), with each requiring | ||
| certain dependencies, and possibly hardware. | ||
|
|
||
| ## Native Sources | ||
|
|
||
| There are two ways to run blueprints with [Native Sources](/developers/deployment/sources/native) in the | ||
| `blueprint-manager`: sandboxed (**recommended**) or not | ||
|
|
||
| In either case, the following dependencies will be needed: | ||
|
|
||
| - [GitHub CLI] (for binary attestations) | ||
|
|
||
| Then additionally for the two execution methods: | ||
|
|
||
| ### Not Sandboxed | ||
|
|
||
| No extra dependencies, the blueprint will run as a normal host process. | ||
|
|
||
| ### Sandboxed (**Linux Only**) | ||
|
|
||
| - [cloud-hypervisor] | ||
| - Note, no additional setup of `cloud-hypervisor` needs to be done. The manager handles downloading the latest kernel | ||
| and disk images. Simply installing and adding it to `PATH` is enough. | ||
| - Allow `CAP_NET_ADMIN` for the `blueprint-manager` binary | ||
| - This can be done by running `setcap cap_net_admin+eip /path/to/blueprint-manager` | ||
| - **_or_** simply running the `blueprint-manager` as root (**not recommended**) | ||
|
|
||
| ## Container Sources | ||
|
|
||
| The requirements for running blueprints with [Container Sources](/developers/deployment/sources/container) are: | ||
|
|
||
| - [Kubernetes] | ||
| - [Docker] | ||
| - The [Kata Containers] runtime | ||
|
|
||
| ## TEE Sources (WIP, **Linux Only**) | ||
|
|
||
| The requirements for running blueprints with [TEE Sources](/developers/deployment/sources/tee) are: | ||
|
|
||
| - [dstack VMM] | ||
| - TODO? | ||
|
|
||
| ## WASM Sources (WIP) | ||
|
|
||
| The requirements for running blueprints with [WASM Sources](/developers/deployment/sources/wasm) are: | ||
|
|
||
| - TODO | ||
|
|
||
| [GitHub CLI]: https://cli.github.com/ | ||
| [cloud-hypervisor]: https://www.cloudhypervisor.org/ | ||
| [Kubernetes]: https://kubernetes.io/ | ||
| [Docker]: https://www.docker.com/get-started/ | ||
| [Kata Containers]: https://katacontainers.io/ | ||
| [dstack VMM]: https://github.com/Dstack-TEE/dstack/tree/master?tab=readme-ov-file#-getting-started |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no, but it's fine for illustration to show that it's only a part of the
blueprint.json