SDK image = runtime image + development utilities #194

endersonmaia · 2025-03-18T20:11:40Z

This pull request includes several changes to the @cartesi/sdk package, focusing on updating Docker images, adding new workflows, and modifying the Dockerfile. The most important changes include adding new container image releases, updating base images, creating databases with migrations, and adding new GitHub workflows for building and pushing Docker images.

Updates to Docker images and base images:

Updated the Debian base image to bookworm-20250317-slim in .changeset/honest-spoons-appear.md and packages/sdk/docker-bake.hcl. [1] [2]
Added new container image releases for cartesi/rollups-database and updated the Dockerfile to use new base images for cartesi, espresso-dev-node, and postgres. [1] [2] [3] [4] [5]

Database migrations:

Created databases and ran migrations for a preinitialized PostgreSQL database in .changeset/hot-timers-repair.md.
Modified the Dockerfile to include scripts for creating and migrating databases for rollupsdb, graphql, and espresso.

New GitHub workflows:

Added rollups-database.yml and runtime.yml workflows to build and push Docker images for rollups-database and sdk-node. [1] [2]

Dockerfile modifications:

Refactored the Dockerfile to use new ARGs for base images and added steps to install necessary packages and set up the environment. [1] [2] [3] [4] [5]

Docker bake configuration:

Updated docker-bake.hcl and docker-bake.override.hcl to include new targets for runtime and database, and set appropriate tags for the images. [1] [2]

changeset-bot · 2025-03-18T20:11:43Z

🦋 Changeset detected

Latest commit: b842d98

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@cartesi/sdk	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

packages/sdk/Dockerfile

tuler

I think the order should be something like:

FROM debian as runtime
# add cartesi-machine
# add cartesi-rollups-node
# add graphql
# add espresso-reader

FROM runtime as sdk
# add anvil
# add devnet
# add crane
# add genext2fsx
# add migrations

.github/workflows/rollups-node.yml

packages/sdk/docker-bake.hcl

tuler · 2025-03-20T22:28:16Z

packages/sdk/docker-bake.override.hcl

 }
+
+target "rollups-node" {
+  tags = ["cartesi/sdk-node:devel"]


let's discuss a bit more on this name

packages/sdk/Dockerfile

endersonmaia · 2025-03-21T12:42:13Z

I think the order should be something like:

FROM debian as runtime
# add cartesi-machine
# add cartesi-rollups-node
# add graphql
# add espresso-reader

FROM runtime as sdk
# add anvil
# add devnet
# add crane
# add genext2fsx
# add migrations

Yeah, I decided to use node:slim as the base for the sdk to reduce the size a little bit.

The order you suggested makes the code more clean at the expense of some MB in the final sdk image.

endersonmaia · 2025-03-24T12:58:37Z

I think the order should be something like:
FROM debian as runtime
# add cartesi-machine
# add cartesi-rollups-node
# add graphql
# add espresso-reader

FROM runtime as sdk
# add anvil
# add devnet
# add crane
# add genext2fsx
# add migrations
Yeah, I decided to use node:slim as the base for the sdk to reduce the size a little bit.

The order you suggested makes the code more clean at the expense of some MB in the final sdk image.

Just accepted your suggestion after evaluating the size difference was negligible.

endersonmaia · 2025-03-24T13:53:53Z

We still need to define the naming, and we make the final fixups to stick with whatever naming is decided.

tuler · 2025-03-24T15:00:15Z

.github/workflows/rollups-database.yaml

@@ -0,0 +1,91 @@
+name: rollups-database


use database and database.yaml to follow name convention.

packages/sdk/Dockerfile

tuler · 2025-03-24T15:36:37Z

packages/sdk/Dockerfile

+    cartesi
+EOF
+
+RUN mkdir -p /tmp/.cartesi && chmod 1777 /tmp/.cartesi


I think we don't need this anymore

packages/sdk/Dockerfile

tuler · 2025-03-24T15:38:26Z

packages/sdk/Dockerfile

+EOF
+
+COPY --from=runtime /usr/bin/cartesi-rollups-cli /usr/bin/
+COPY --from=runtime /usr/lib/libcartesi* /usr/lib/


is this safe? why is it needed?

We need this to run cartesi-rollups-cli without the need to reinstall its package.

tuler · 2025-03-24T15:40:33Z

packages/sdk/Dockerfile

 USER root
 ARG DEBIAN_FRONTEND=noninteractive
 RUN <<EOF
+apt-get update


some of the packages below were already installed by runtime base

tuler · 2025-03-24T15:44:04Z

Here is a commit organization that may be better to review and merge:

A commit with only the Dockerfile. No other change is necessary to the workflows, as the final default target is the sdk as it is today. That is already mergable by itself.

Then we handle the case for database and runtime on top of that.
I have an idea for the workflow that might simplify it.

A container image will be released only with the binaries needed to run a rollups-node with optional auxiliary services like rollups-graphql and espresso-reader.

This container image will be based on postgres and contains a pre-initialized database with rollups-node, rollups-graphql and espresso-reader database migrations ready to be used.

endersonmaia · 2025-03-24T16:25:36Z

A commit with only the Dockerfile. No other change is necessary to the workflows, as the final default target is the sdk as it is today. That is already mergable by itself.

There's a PR attempt for that at #207

endersonmaia · 2025-03-24T16:54:32Z

Then we handle the case for database and runtime on top of that.

Another PR for that: #209

tuler · 2025-03-25T01:57:22Z

Superseded by #209

tuler reviewed Mar 18, 2025

View reviewed changes

packages/sdk/Dockerfile Outdated Show resolved Hide resolved

tuler changed the title ~~Feature/sdk extra images~~ SDK image = runtime image + development utilities Mar 18, 2025

endersonmaia force-pushed the feature/sdk-extra-images branch from 1129770 to 1c04a25 Compare March 20, 2025 19:38

endersonmaia requested a review from tuler March 20, 2025 19:44

endersonmaia marked this pull request as ready for review March 20, 2025 19:52

endersonmaia self-assigned this Mar 20, 2025

endersonmaia added the sdk label Mar 20, 2025

tuler reviewed Mar 20, 2025

View reviewed changes

tuler linked an issue Mar 20, 2025 that may be closed by this pull request

Runtime and SDK images #200

Closed

3 tasks

endersonmaia force-pushed the feature/sdk-extra-images branch from 823490b to cdfc3c6 Compare March 21, 2025 13:43

endersonmaia mentioned this pull request Mar 21, 2025

cli to use pre-initialized database #205

Merged

endersonmaia force-pushed the feature/sdk-extra-images branch from 5343b63 to 2c9ae08 Compare March 24, 2025 13:49

endersonmaia requested a review from tuler March 24, 2025 13:53

tuler reviewed Mar 24, 2025

View reviewed changes

endersonmaia added 7 commits March 24, 2025 13:00

chore(sdk): bump Debian base image to bookworm-20250317-slim

1e08f57

feat(sdk): add cartesi/rollups-node container release

bcb32b2

A container image will be released only with the binaries needed to run a rollups-node with optional auxiliary services like rollups-graphql and espresso-reader.

refactor(sdk): add CARTESI_ prefix to variables

68b97f9

refactor(sdk): convention _BASE_IMAGE prefixed ARGs with full image name

85698d8

feat(sdk): pin container image digests

d499b41

refactor(sdk): single base image

3f5c0fd

feat(sdk): add cartesi/rollups-database container release

113129b

This container image will be based on postgres and contains a pre-initialized database with rollups-node, rollups-graphql and espresso-reader database migrations ready to be used.

endersonmaia force-pushed the feature/sdk-extra-images branch from 1ae64db to 113129b Compare March 24, 2025 16:00

endersonmaia added 2 commits March 24, 2025 13:13

fixup! feat(sdk): add cartesi/rollups-database container release

f204d67

fixup! feat(sdk): add cartesi/rollups-node container release

b842d98

tuler closed this Mar 25, 2025

SDK image = runtime image + development utilities #194

SDK image = runtime image + development utilities #194

Uh oh!

Conversation

endersonmaia commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Updates to Docker images and base images:

Database migrations:

New GitHub workflows:

Dockerfile modifications:

Docker bake configuration:

Uh oh!

changeset-bot bot commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Uh oh!

tuler left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tuler Mar 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

endersonmaia commented Mar 21, 2025

Uh oh!

endersonmaia commented Mar 24, 2025

Uh oh!

endersonmaia commented Mar 24, 2025

Uh oh!

tuler Mar 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tuler Mar 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tuler Mar 24, 2025

Choose a reason for hiding this comment

Uh oh!

endersonmaia Mar 24, 2025

Choose a reason for hiding this comment

Uh oh!

tuler Mar 24, 2025

Choose a reason for hiding this comment

Uh oh!

tuler commented Mar 24, 2025

Uh oh!

endersonmaia commented Mar 24, 2025

Uh oh!

endersonmaia commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tuler commented Mar 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

endersonmaia commented Mar 18, 2025 •

edited

Loading

changeset-bot bot commented Mar 18, 2025 •

edited

Loading

endersonmaia commented Mar 24, 2025 •

edited

Loading