From 0a6c26b25b0346a1c8aecc5eaa3f1f858017becc Mon Sep 17 00:00:00 2001 From: Yu Hu Date: Tue, 23 Dec 2025 10:00:40 -0500 Subject: [PATCH 01/11] documentation --- README.md | 123 ++++++++++- features/src/ttyd/README.md | 39 ++++ features/src/ttyd/devcontainer-feature.json | 19 ++ features/src/ttyd/install.sh | 73 +++++++ scripts/create-custom-app.sh | 206 +++++++++++++++++++ src/{jupyter => example}/.devcontainer.json | 14 +- src/example/README.md | 34 +++ src/example/devcontainer-template.json | 37 ++++ src/{jupyter => example}/docker-compose.yaml | 8 +- src/jupyter/README.md | 17 -- src/jupyter/devcontainer-template.json | 23 --- 11 files changed, 538 insertions(+), 55 deletions(-) create mode 100644 features/src/ttyd/README.md create mode 100644 features/src/ttyd/devcontainer-feature.json create mode 100755 features/src/ttyd/install.sh create mode 100755 scripts/create-custom-app.sh rename src/{jupyter => example}/.devcontainer.json (87%) create mode 100644 src/example/README.md create mode 100644 src/example/devcontainer-template.json rename src/{jupyter => example}/docker-compose.yaml (81%) delete mode 100644 src/jupyter/README.md delete mode 100644 src/jupyter/devcontainer-template.json diff --git a/README.md b/README.md index 06ab06d7..187daae2 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,17 @@ # workbench-app-devcontainer -Repo to store Verily Workbench-specific applications' devcontainer specifications. To develop your own custom app configuration, clone this repo. +Repo to store Verily Workbench-specific applications' devcontainer specifications. To develop your own custom app configuration, fork this repo. + +## Repository Structure + +- **`src/`**: Contains devcontainer app templates for various applications (Jupyter, R/RStudio, VSCode, etc.) + - Each subdirectory represents a complete app template with `.devcontainer.json`, `docker-compose.yaml`, and startup scripts + - Example: `src/example/` - A reference implementation showing the basic structure +- **`features/src/`**: Contains reusable devcontainer features that can be included in app templates + - `workbench-tools/` - Bioinformatics tools (plink, plink2, regenie, bcftools, samtools, etc.) + - `java/`, `jupyter/` - Language/framework-specific features +- **`startupscript/`**: VM provisioning scripts that run after container creation +- **`test/`**: Integration tests to verify app templates ## Workbench-specific application requirements @@ -12,11 +23,117 @@ Repo to store Verily Workbench-specific applications' devcontainer specification https://containers.dev/ +## Developing a New App + +To create a custom app for Workbench: + +1. **Fork this repository** to your own GitHub account or organization + +2. **Create a new directory** under `src/` for your app (e.g., `src/my-custom-app/`) + +3. **Add required files** to your app directory: + - `.devcontainer.json` - The devcontainer specification that defines your app configuration + - `docker-compose.yaml` - Docker Compose configuration (must follow Workbench requirements above) + - `startup.sh` - App-specific startup script (if needed) + +4. **Configure your `.devcontainer.json`**: + + At a bare minimum, you need to specify: + + - **Docker image**: The base container image your app runs on (e.g., `jupyter/base-notebook`, `rocker/rstudio`) + - **Port**: The port your application exposes (e.g., `8888` for Jupyter, `8787` for RStudio). This port is exposed on the bridge network so Workbench can reach your app + - **Default user**: The username that your application runs as inside the container (e.g., `jovyan` for Jupyter, `rstudio` for RStudio). If your app doesn't have a specific user, you can use `root` + - **Home directory**: The default working directory for the user. In most cases, this is `/home/$(whoami)` (e.g., `/home/jovyan`, `/home/rstudio`). If the default user is `root`, the home directory is typically `/root`. Note: VSCode is a unique case where the home directory is `/config` + + **Important**: The home directory is where Workbench mounts cloud storage buckets and clones GitHub repositories. These will be located at: + - Cloud storage buckets: `${homedir}/workspaces` + - GitHub repositories: `${homedir}/repos` + + Additional configuration: + - Set `postCreateCommand` to run `post-startup.sh` with parameters: `[username, home_dir, ${templateOption:cloud}]` + - Include any needed features from `features/src/` (e.g., `workbench-tools`) + - Use template option `${templateOption:cloud}` to specify the cloud provider (GCP or AWS) + + You can use the script `./scripts/create-custom-app.sh` to generate a basic devcontainer structure from these parameters. + +5. **Test your app**: + - Run the test script: `cd test && ./test.sh ` + - Create a custom app in Workbench UI pointing to your forked repo and branch + +6. **Reference the example app** at [src/example](https://github.com/verily-src/workbench-app-devcontainer/tree/main/src/example) to see a basic implementation + +For detailed guidance, visit https://support.workbench.verily.com/docs/guides/cloud_apps/create_custom_apps/ + +## Running Linux Distros on Workbench + +Linux distributions (Ubuntu, Debian, RHEL, etc.) typically don't have a web UI or exposed port by default. Since Workbench apps must be accessible via a browser, you need to add a web-based interface to your Linux distro container. + +### Recommended Approaches + +#### Option 1: JupyterLab (Recommended for Data Science & General Use) + +JupyterLab provides a full-featured web interface with built-in terminal access, file browser, text editor, and notebook support. + +**Examples**: See the NeMo and Parabricks apps (`src/nemo_jupyter/` and `src/workbench-jupyter-parabricks/`) which use JupyterLab to provide web access to specialized NVIDIA CUDA Linux environments. + +To add JupyterLab to your Linux distro, use the `features/src/jupyter` feature with `installJupyterlab: true` and configure the container command: + +```yaml +# docker-compose.yaml +services: + app: + container_name: "application-server" + image: "ubuntu:22.04" + command: ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--LabApp.token=''"] + ports: + - 8888:8888 + # ... rest of configuration +``` + +#### Option 2: ttyd (Lightweight Terminal-Only Access) + +If you only need terminal access without the full JupyterLab interface, use [ttyd](https://github.com/tsl0922/ttyd) - a lightweight web-based terminal. + +To add ttyd to your Linux distro, use the `features/src/ttyd` feature and configure the container command: + +```yaml +# docker-compose.yaml +services: + app: + container_name: "application-server" + image: "ubuntu:22.04" + command: ["ttyd", "-p", "7681", "bash"] + ports: + - 7681:7681 + # ... rest of configuration +``` + +#### Option 3: VS Code Server (Full IDE Experience) + +For a full IDE experience, use the [vscode-server feature](https://github.com/devcontainers-extra/features/tree/main/src/vscode-server) which provides VS Code in the browser with built-in terminal access. + +## Debugging and Local Development + +To run and debug your app locally: + +1. **Install the devcontainer CLI**: Follow the installation instructions at https://code.visualstudio.com/docs/devcontainers/devcontainer-cli + +2. **Build your app**: + ```bash + cd src/ + devcontainer build --workspace-folder . + ``` + +3. **Start your app**: + ```bash + devcontainer up --workspace-folder . + ``` + +4. **Access your app**: Once the container is running, you can access it at `localhost:` where `` is the port you specified in your configuration (e.g., `localhost:8888` for Jupyter) + ## How to use The `.devcontainer.json` file in the custom app folder (e.g. r-analysis/) contains the custom app configuration. `post-startup.sh` contains workbench specific set up. Please visit https://support.workbench.verily.com/docs/guides/cloud_apps/create_custom_apps/ for details about using a dev container specification to create a custom app in Workbench. - -For an example app, see [src/jupyter](https://github.com/verily-src/workbench-app-devcontainer/tree/main/src/jupyter). diff --git a/features/src/ttyd/README.md b/features/src/ttyd/README.md new file mode 100644 index 00000000..bb8a3ce8 --- /dev/null +++ b/features/src/ttyd/README.md @@ -0,0 +1,39 @@ +# ttyd - Web-based Terminal + +Installs [ttyd](https://github.com/tsl0922/ttyd), a simple command-line tool for sharing your terminal over the web. + +## Usage + +```json +"features": { + "./.devcontainer/features/ttyd": { + "version": "latest", + "port": "7681" + } +} +``` + +## Options + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| version | string | `latest` | Version of ttyd to install (e.g., '1.7.7' or 'latest') | +| port | string | `7681` | Port ttyd will listen on | + +## Running ttyd + +Configure ttyd as the container command in your `docker-compose.yaml`: + +```yaml +services: + app: + command: ["ttyd", "-p", "7681", "bash"] + ports: + - 7681:7681 +``` + +Then access the terminal in your browser at `http://localhost:7681`. + +## Example + +See the repository README for examples of using ttyd with Linux distributions that don't have a built-in UI. diff --git a/features/src/ttyd/devcontainer-feature.json b/features/src/ttyd/devcontainer-feature.json new file mode 100644 index 00000000..f6673811 --- /dev/null +++ b/features/src/ttyd/devcontainer-feature.json @@ -0,0 +1,19 @@ +{ + "id": "ttyd", + "name": "ttyd - Web-based Terminal", + "version": "1.0.0", + "description": "Installs ttyd, a simple command-line tool for sharing terminal over the web", + "documentationURL": "https://github.com/tsl0922/ttyd", + "options": { + "version": { + "type": "string", + "default": "latest", + "description": "Version of ttyd to install (e.g., '1.7.7' or 'latest')" + }, + "port": { + "type": "string", + "default": "7681", + "description": "Port ttyd will listen on" + } + } +} diff --git a/features/src/ttyd/install.sh b/features/src/ttyd/install.sh new file mode 100755 index 00000000..774a2c2c --- /dev/null +++ b/features/src/ttyd/install.sh @@ -0,0 +1,73 @@ +#!/bin/bash +set -o errexit -o nounset -o pipefail -o xtrace + +# ttyd feature install script +# Installs ttyd - a simple tool for sharing terminal over the web + +readonly VERSION="${VERSION:-"latest"}" +readonly PORT="${PORT:-"7681"}" + +echo "Installing ttyd ${VERSION}..." + +# Check for root +if [ "$(id -u)" -ne 0 ]; then + echo 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.' + exit 1 +fi + +# Detect OS +. /etc/os-release +if [ "${ID}" = "debian" ] || [ "${ID_LIKE}" = "debian" ]; then + ADJUSTED_ID="debian" +elif [[ "${ID}" = "rhel" || "${ID}" = "fedora" || "${ID_LIKE}" = *"rhel"* || "${ID_LIKE}" = *"fedora"* ]]; then + ADJUSTED_ID="rhel" +else + echo "Linux distro ${ID} not supported." + exit 1 +fi + +# Install dependencies +if [ "${ADJUSTED_ID}" = "debian" ]; then + apt-get update + apt-get install -y --no-install-recommends curl ca-certificates +elif [ "${ADJUSTED_ID}" = "rhel" ]; then + yum install -y curl ca-certificates +fi + +# Determine architecture +ARCH="$(uname -m)" +case "${ARCH}" in + x86_64) ARCH="x86_64" ;; + aarch64) ARCH="aarch64" ;; + armv7l) ARCH="armv7" ;; + *) echo "Unsupported architecture: ${ARCH}"; exit 1 ;; +esac + +# Get latest version if needed +if [ "${VERSION}" = "latest" ]; then + TTYD_VERSION=$(curl -sL https://api.github.com/repos/tsl0922/ttyd/releases/latest | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/') +else + TTYD_VERSION="${VERSION}" +fi + +echo "Installing ttyd version ${TTYD_VERSION} for ${ARCH}..." + +# Download and install ttyd +DOWNLOAD_URL="https://github.com/tsl0922/ttyd/releases/download/${TTYD_VERSION}/ttyd.${ARCH}" +curl -sL "${DOWNLOAD_URL}" -o /usr/local/bin/ttyd +chmod +x /usr/local/bin/ttyd + +# Verify installation +if ! ttyd --version; then + echo "Failed to install ttyd" + exit 1 +fi + +# Clean up +if [ "${ADJUSTED_ID}" = "debian" ]; then + rm -rf /var/lib/apt/lists/* +fi + +echo "ttyd ${TTYD_VERSION} installed successfully!" +echo "Default port: ${PORT}" +echo "To run: ttyd -p ${PORT} bash" diff --git a/scripts/create-custom-app.sh b/scripts/create-custom-app.sh new file mode 100755 index 00000000..c20ad6aa --- /dev/null +++ b/scripts/create-custom-app.sh @@ -0,0 +1,206 @@ +#!/bin/bash +# Script to create a custom Workbench app devcontainer structure +# Usage: ./create-custom-app.sh [username] [home-dir] + +set -o errexit -o nounset -o pipefail -o xtrace + +# Parse arguments +if [ $# -lt 3 ]; then + echo "Usage: $0 [username] [home-dir]" + echo "" + echo "Arguments:" + echo " app-name - Name of your custom app (e.g., my-jupyter-app)" + echo " docker-image - Docker image to use (e.g., jupyter/base-notebook)" + echo " port - Port your app exposes (e.g., 8888)" + echo " username - (Optional) User inside container (default: root)" + echo " home-dir - (Optional) Home directory (default: /root or /home/)" + echo "" + echo "Example:" + echo " $0 my-jupyter jupyter/base-notebook 8888 jovyan /home/jovyan" + exit 1 +fi + +readonly APP_NAME="$1" +readonly DOCKER_IMAGE="$2" +readonly PORT="$3" +readonly USERNAME="${4:-root}" + +# Calculate home directory if not provided +if [ $# -ge 5 ]; then + readonly HOME_DIR="$5" +else + if [ "$USERNAME" = "root" ]; then + readonly HOME_DIR="/root" + else + readonly HOME_DIR="/home/$USERNAME" + fi +fi +readonly -f + +readonly APP_DIR="src/${APP_NAME}" +readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +readonly REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" + +# Create app directory +echo "Creating app directory: ${APP_DIR}" +mkdir -p "${REPO_ROOT}/${APP_DIR}" + +# Generate .devcontainer.json +echo "Generating .devcontainer.json" +cat > "${REPO_ROOT}/${APP_DIR}/.devcontainer.json" < "${REPO_ROOT}/${APP_DIR}/docker-compose.yaml" < "${REPO_ROOT}/${APP_DIR}/devcontainer-template.json" < "${REPO_ROOT}/${APP_DIR}/README.md" < Date: Tue, 23 Dec 2025 10:29:41 -0500 Subject: [PATCH 02/11] add test --- .github/workflows/test-pr.yaml | 2 - .github/workflows/test-scripts.yaml | 43 +++++++ features/src/ttyd/install.sh | 27 +++-- features/test/ttyd/README.md | 68 +++++++++++ features/test/ttyd/scenarios.json | 24 ++++ features/test/ttyd/test.sh | 26 +++++ scripts/test/README.md | 84 ++++++++++++++ scripts/test/create-custom-app.bats | 171 ++++++++++++++++++++++++++++ 8 files changed, 432 insertions(+), 13 deletions(-) create mode 100644 .github/workflows/test-scripts.yaml create mode 100644 features/test/ttyd/README.md create mode 100644 features/test/ttyd/scenarios.json create mode 100755 features/test/ttyd/test.sh create mode 100644 scripts/test/README.md create mode 100755 scripts/test/create-custom-app.bats diff --git a/.github/workflows/test-pr.yaml b/.github/workflows/test-pr.yaml index f24f6431..e48c6064 100644 --- a/.github/workflows/test-pr.yaml +++ b/.github/workflows/test-pr.yaml @@ -27,8 +27,6 @@ jobs: config: - jupyter-template: user: jupyter - jupyter: - user: jovyan r-analysis: user: rstudio workbench_tools: true diff --git a/.github/workflows/test-scripts.yaml b/.github/workflows/test-scripts.yaml new file mode 100644 index 00000000..215bf11e --- /dev/null +++ b/.github/workflows/test-scripts.yaml @@ -0,0 +1,43 @@ +name: "CI - Test Scripts and Features" +on: + pull_request: + paths: + - 'scripts/**' + - 'features/**' + - '.github/workflows/test-scripts.yaml' + push: + branches: + - master + paths: + - 'scripts/**' + - 'features/**' + - '.github/workflows/test-scripts.yaml' + +jobs: + test-create-custom-app: + name: "Test create-custom-app.sh" + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Install bats-core + run: | + sudo apt-get update + sudo apt-get install -y bats + + - name: Run bats tests + run: | + cd scripts/test + bats create-custom-app.bats + + test-ttyd-feature: + name: "Test ttyd feature" + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Install devcontainer CLI + run: npm install -g @devcontainers/cli + + - name: Run ttyd feature tests + run: devcontainer features test --features ttyd --base-image mcr.microsoft.com/devcontainers/base:ubuntu . diff --git a/features/src/ttyd/install.sh b/features/src/ttyd/install.sh index 774a2c2c..13917af2 100755 --- a/features/src/ttyd/install.sh +++ b/features/src/ttyd/install.sh @@ -4,10 +4,10 @@ set -o errexit -o nounset -o pipefail -o xtrace # ttyd feature install script # Installs ttyd - a simple tool for sharing terminal over the web -readonly VERSION="${VERSION:-"latest"}" -readonly PORT="${PORT:-"7681"}" +readonly TTYD_VERSION="${VERSION:-"latest"}" +readonly TTYD_PORT="${PORT:-"7681"}" -echo "Installing ttyd ${VERSION}..." +echo "Installing ttyd ${TTYD_VERSION}..." # Check for root if [ "$(id -u)" -ne 0 ]; then @@ -44,16 +44,21 @@ case "${ARCH}" in esac # Get latest version if needed -if [ "${VERSION}" = "latest" ]; then - TTYD_VERSION=$(curl -sL https://api.github.com/repos/tsl0922/ttyd/releases/latest | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/') +if [ "${TTYD_VERSION}" = "latest" ]; then + TTYD_RELEASE_VERSION=$(curl -sL https://api.github.com/repos/tsl0922/ttyd/releases/latest | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/' || echo "") + # Fallback to a known version if API call fails + if [ -z "${TTYD_RELEASE_VERSION}" ]; then + echo "Warning: Could not fetch latest version from GitHub API, using fallback version 1.7.7" + TTYD_RELEASE_VERSION="1.7.7" + fi else - TTYD_VERSION="${VERSION}" + TTYD_RELEASE_VERSION="${TTYD_VERSION}" fi -echo "Installing ttyd version ${TTYD_VERSION} for ${ARCH}..." +echo "Installing ttyd version ${TTYD_RELEASE_VERSION} for ${ARCH}..." # Download and install ttyd -DOWNLOAD_URL="https://github.com/tsl0922/ttyd/releases/download/${TTYD_VERSION}/ttyd.${ARCH}" +DOWNLOAD_URL="https://github.com/tsl0922/ttyd/releases/download/${TTYD_RELEASE_VERSION}/ttyd.${ARCH}" curl -sL "${DOWNLOAD_URL}" -o /usr/local/bin/ttyd chmod +x /usr/local/bin/ttyd @@ -68,6 +73,6 @@ if [ "${ADJUSTED_ID}" = "debian" ]; then rm -rf /var/lib/apt/lists/* fi -echo "ttyd ${TTYD_VERSION} installed successfully!" -echo "Default port: ${PORT}" -echo "To run: ttyd -p ${PORT} bash" +echo "ttyd ${TTYD_RELEASE_VERSION} installed successfully!" +echo "Default port: ${TTYD_PORT}" +echo "To run: ttyd -p ${TTYD_PORT} bash" diff --git a/features/test/ttyd/README.md b/features/test/ttyd/README.md new file mode 100644 index 00000000..b60dac04 --- /dev/null +++ b/features/test/ttyd/README.md @@ -0,0 +1,68 @@ +# ttyd Feature Tests + +This directory contains tests for the `ttyd` devcontainer feature. + +## Test Structure + +Following the [devcontainer feature testing guidelines](https://github.com/devcontainers/cli/blob/main/docs/features/test.md), this directory contains: + +- **`scenarios.json`**: Defines test scenarios with different feature configurations +- **`test.sh`**: Test script that validates ttyd installation and functionality + +## Test Scenarios + +### ttyd-default +Tests ttyd installation with default options (latest version, default port 7681) + +### ttyd-specific-version +Tests ttyd installation with a specific version (1.7.7) + +### ttyd-custom-port +Tests ttyd installation with a custom port (8080) + +## Running Tests Locally + +### Prerequisites + +Install the devcontainer CLI: +```bash +npm install -g @devcontainers/cli +``` + +### Run All Tests + +From the `features` directory: +```bash +cd features +devcontainer features test --features ttyd --base-image mcr.microsoft.com/devcontainers/base:ubuntu . +``` + +### Run Specific Scenario + +```bash +cd features +devcontainer features test \ + --features ttyd \ + --base-image mcr.microsoft.com/devcontainers/base:ubuntu \ + --filter ttyd-default \ + . +``` + +**Note**: Running tests locally on macOS with Colima may encounter Docker mount issues. The tests will run properly in Linux environments and in CI. + +## What the Tests Verify + +The test script (`test.sh`) verifies: + +1. ✅ ttyd is installed and in PATH +2. ✅ ttyd version command works +3. ✅ ttyd version output matches expected format +4. ✅ ttyd help command works + +## CI/CD Integration + +These tests run automatically in GitHub Actions when: +- A pull request modifies `features/**` +- Changes are pushed to the master branch + +See `.github/workflows/test-scripts.yaml` for the CI configuration. diff --git a/features/test/ttyd/scenarios.json b/features/test/ttyd/scenarios.json new file mode 100644 index 00000000..6a4fa3f1 --- /dev/null +++ b/features/test/ttyd/scenarios.json @@ -0,0 +1,24 @@ +{ + "ttyd-default": { + "image": "mcr.microsoft.com/devcontainers/base:ubuntu", + "features": { + "ttyd": {} + } + }, + "ttyd-specific-version": { + "image": "mcr.microsoft.com/devcontainers/base:ubuntu", + "features": { + "ttyd": { + "version": "1.7.7" + } + } + }, + "ttyd-custom-port": { + "image": "mcr.microsoft.com/devcontainers/base:ubuntu", + "features": { + "ttyd": { + "port": "8080" + } + } + } +} diff --git a/features/test/ttyd/test.sh b/features/test/ttyd/test.sh new file mode 100755 index 00000000..4ca60334 --- /dev/null +++ b/features/test/ttyd/test.sh @@ -0,0 +1,26 @@ +#!/bin/bash + +# This test file will be executed against an auto-generated devcontainer.json that +# includes the 'ttyd' feature with various options. + +set -e + +# Optional: Import test library bundled with the devcontainer CLI +# See https://github.com/devcontainers/cli/blob/HEAD/docs/features/test.md#dev-container-features-test-lib +# Provides the 'check' command to execute tests and the 'reportResults' function to report results. +source dev-container-features-test-lib + +# Feature-specific tests +# The 'check' command takes a label and a command to run. + +check "ttyd installed" which ttyd + +check "ttyd version" ttyd --version + +check "ttyd executable" bash -c "ttyd --version 2>&1 | grep -E 'ttyd version [0-9]+\.[0-9]+\.[0-9]+'" + +check "ttyd help" ttyd --help + +# Report result +# If any of the checks above exited with a non-zero exit code, the test will fail. +reportResults diff --git a/scripts/test/README.md b/scripts/test/README.md new file mode 100644 index 00000000..762975ef --- /dev/null +++ b/scripts/test/README.md @@ -0,0 +1,84 @@ +# Script Tests + +This directory contains tests for scripts in the `scripts/` directory. + +## Prerequisites + +Install [bats-core](https://github.com/bats-core/bats-core) to run the tests: + +```bash +# On macOS with Homebrew +brew install bats-core + +# On Ubuntu/Debian +sudo apt-get install bats + +# Or install from source +git clone https://github.com/bats-core/bats-core.git +cd bats-core +sudo ./install.sh /usr/local +``` + +## Running Tests + +### Run all tests + +```bash +cd scripts/test +bats . +``` + +### Run a specific test file + +```bash +bats scripts/test/create-custom-app.bats +``` + +### Run a specific test case + +```bash +bats scripts/test/create-custom-app.bats --filter "shows usage" +``` + +## Test Coverage + +### create-custom-app.bats + +Tests for the `create-custom-app.sh` script: + +- ✅ Usage/help message validation +- ✅ Minimal arguments (defaults to root user) +- ✅ Custom username and home directory +- ✅ Generated `.devcontainer.json` structure +- ✅ Generated `docker-compose.yaml` structure +- ✅ Generated `devcontainer-template.json` with correct defaults +- ✅ Generated README.md content +- ✅ Home directory defaults (/root for root, /home/username otherwise) +- ✅ Valid JSON output +- ✅ Success message output + +## Writing New Tests + +Follow the bats format: + +```bash +@test "description of test" { + run ./your-script.sh args + [ "$status" -eq 0 ] + [[ "$output" == *"expected string"* ]] +} +``` + +Use the `setup()` and `teardown()` functions to manage test state: + +```bash +setup() { + # Runs before each test + TEST_TEMP_DIR="$(mktemp -d)" +} + +teardown() { + # Runs after each test + rm -rf "${TEST_TEMP_DIR}" +} +``` diff --git a/scripts/test/create-custom-app.bats b/scripts/test/create-custom-app.bats new file mode 100755 index 00000000..f267b000 --- /dev/null +++ b/scripts/test/create-custom-app.bats @@ -0,0 +1,171 @@ +#!/usr/bin/env bats +# Test suite for create-custom-app.sh script + +setup() { + # Get the directory of this test file + DIR="$( cd "$( dirname "$BATS_TEST_FILENAME" )" >/dev/null 2>&1 && pwd )" + REPO_ROOT="$(cd "${DIR}/../.." && pwd)" + SCRIPT="${REPO_ROOT}/scripts/create-custom-app.sh" + + export REPO_ROOT + export SCRIPT + + # Work from repo root so script can create files correctly + cd "${REPO_ROOT}" +} + +teardown() { + # Clean up any test apps created in src/ + if [ -d "${REPO_ROOT}/src/test-app" ]; then + rm -rf "${REPO_ROOT}/src/test-app" + fi + if [ -d "${REPO_ROOT}/src/my-jupyter" ]; then + rm -rf "${REPO_ROOT}/src/my-jupyter" + fi +} + +@test "create-custom-app.sh: shows usage when no arguments provided" { + run bash "${SCRIPT}" + [ "$status" -eq 1 ] + [[ "$output" == *"Usage:"* ]] + [[ "$output" == *"app-name"* ]] + [[ "$output" == *"docker-image"* ]] + [[ "$output" == *"port"* ]] +} + +@test "create-custom-app.sh: shows usage when insufficient arguments provided" { + run bash "${SCRIPT}" my-app + [ "$status" -eq 1 ] + [[ "$output" == *"Usage:"* ]] +} + +@test "create-custom-app.sh: creates app with minimal arguments (defaults to root user)" { + run bash "${SCRIPT}" test-app python:3.11 8080 + [ "$status" -eq 0 ] + [ -d "src/test-app" ] + [ -f "src/test-app/.devcontainer.json" ] + [ -f "src/test-app/docker-compose.yaml" ] + [ -f "src/test-app/devcontainer-template.json" ] + [ -f "src/test-app/README.md" ] +} + +@test "create-custom-app.sh: creates app with custom username and home directory" { + run bash "${SCRIPT}" my-jupyter jupyter/base-notebook 8888 jovyan /home/jovyan + [ "$status" -eq 0 ] + [ -d "src/my-jupyter" ] +} + +@test "create-custom-app.sh: .devcontainer.json contains correct template variables" { + bash "${SCRIPT}" test-app python:3.11 8080 testuser /home/testuser + + # Check that .devcontainer.json has the correct structure + [ -f "src/test-app/.devcontainer.json" ] + + # Verify it contains template options + grep -q '${templateOption:username}' "src/test-app/.devcontainer.json" + grep -q '${templateOption:homeDir}' "src/test-app/.devcontainer.json" + grep -q '${templateOption:cloud}' "src/test-app/.devcontainer.json" + + # Verify postCreateCommand exists + grep -q 'postCreateCommand' "src/test-app/.devcontainer.json" + + # Verify postStartCommand exists + grep -q 'postStartCommand' "src/test-app/.devcontainer.json" +} + +@test "create-custom-app.sh: docker-compose.yaml contains correct image and port template" { + bash "${SCRIPT}" test-app python:3.11 8080 + + [ -f "src/test-app/docker-compose.yaml" ] + + # Check for template variables + grep -q '${templateOption:image}' "src/test-app/docker-compose.yaml" + grep -q '${templateOption:port}' "src/test-app/docker-compose.yaml" + grep -q '${templateOption:homeDir}' "src/test-app/docker-compose.yaml" + + # Check for required workbench settings + grep -q 'container_name: "application-server"' "src/test-app/docker-compose.yaml" + grep -q 'app-network' "src/test-app/docker-compose.yaml" +} + +@test "create-custom-app.sh: devcontainer-template.json has correct default values" { + bash "${SCRIPT}" my-jupyter jupyter/base-notebook 8888 jovyan /home/jovyan + + [ -f "src/my-jupyter/devcontainer-template.json" ] + + # Check for the app id + grep -q '"id": "my-jupyter"' "src/my-jupyter/devcontainer-template.json" + + # Check for image option with default value + grep -q '"default": "jupyter/base-notebook"' "src/my-jupyter/devcontainer-template.json" + + # Check for port option + grep -q '"default": "8888"' "src/my-jupyter/devcontainer-template.json" + + # Check for username option + grep -q '"default": "jovyan"' "src/my-jupyter/devcontainer-template.json" + + # Check for homeDir option + grep -q '"default": "/home/jovyan"' "src/my-jupyter/devcontainer-template.json" + + # Check for cloud option + grep -q '"cloud"' "src/my-jupyter/devcontainer-template.json" +} + +@test "create-custom-app.sh: README.md is generated with correct content" { + bash "${SCRIPT}" test-app python:3.11 8080 testuser /home/testuser + + [ -f "src/test-app/README.md" ] + + # Check for app name in README + grep -q 'test-app' "src/test-app/README.md" + + # Check for image reference + grep -q 'python:3.11' "src/test-app/README.md" + + # Check for port reference + grep -q '8080' "src/test-app/README.md" + + # Check for username reference + grep -q 'testuser' "src/test-app/README.md" + + # Check for home directory reference + grep -q '/home/testuser' "src/test-app/README.md" +} + +@test "create-custom-app.sh: uses /root as home dir when user is root" { + bash "${SCRIPT}" test-app python:3.11 8080 root + + [ -f "src/test-app/devcontainer-template.json" ] + + # Check that default homeDir is /root for root user + grep -q '"default": "/root"' "src/test-app/devcontainer-template.json" +} + +@test "create-custom-app.sh: uses /home/username as home dir when user is not root" { + bash "${SCRIPT}" test-app python:3.11 8080 myuser + + [ -f "src/test-app/devcontainer-template.json" ] + + # Check that default homeDir is /home/myuser + grep -q '"default": "/home/myuser"' "src/test-app/devcontainer-template.json" +} + +@test "create-custom-app.sh: all generated files are valid JSON" { + bash "${SCRIPT}" test-app python:3.11 8080 + + # Validate .devcontainer.json + run python3 -m json.tool "src/test-app/.devcontainer.json" + [ "$status" -eq 0 ] + + # Validate devcontainer-template.json + run python3 -m json.tool "src/test-app/devcontainer-template.json" + [ "$status" -eq 0 ] +} + +@test "create-custom-app.sh: output message confirms creation" { + run bash "${SCRIPT}" test-app python:3.11 8080 + [ "$status" -eq 0 ] + [[ "$output" == *"Custom app created successfully"* ]] + [[ "$output" == *"src/test-app"* ]] +} From c5f86d4f4435e46e185b46bdd73b420b99fa7217 Mon Sep 17 00:00:00 2001 From: Yu Hu Date: Tue, 23 Dec 2025 10:47:22 -0500 Subject: [PATCH 03/11] clean up --- .github/workflows/test-scripts.yaml | 16 +---- README.md | 13 +++- features/src/ttyd/README.md | 39 ----------- features/src/ttyd/devcontainer-feature.json | 19 ----- features/src/ttyd/install.sh | 78 --------------------- features/test/ttyd/README.md | 68 ------------------ features/test/ttyd/scenarios.json | 24 ------- features/test/ttyd/test.sh | 26 ------- scripts/create-custom-app.sh | 6 +- 9 files changed, 17 insertions(+), 272 deletions(-) delete mode 100644 features/src/ttyd/README.md delete mode 100644 features/src/ttyd/devcontainer-feature.json delete mode 100755 features/src/ttyd/install.sh delete mode 100644 features/test/ttyd/README.md delete mode 100644 features/test/ttyd/scenarios.json delete mode 100755 features/test/ttyd/test.sh diff --git a/.github/workflows/test-scripts.yaml b/.github/workflows/test-scripts.yaml index 215bf11e..49f66be8 100644 --- a/.github/workflows/test-scripts.yaml +++ b/.github/workflows/test-scripts.yaml @@ -1,16 +1,14 @@ -name: "CI - Test Scripts and Features" +name: "CI - Test Scripts" on: pull_request: paths: - 'scripts/**' - - 'features/**' - '.github/workflows/test-scripts.yaml' push: branches: - master paths: - 'scripts/**' - - 'features/**' - '.github/workflows/test-scripts.yaml' jobs: @@ -29,15 +27,3 @@ jobs: run: | cd scripts/test bats create-custom-app.bats - - test-ttyd-feature: - name: "Test ttyd feature" - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - name: Install devcontainer CLI - run: npm install -g @devcontainers/cli - - - name: Run ttyd feature tests - run: devcontainer features test --features ttyd --base-image mcr.microsoft.com/devcontainers/base:ubuntu . diff --git a/README.md b/README.md index 187daae2..b219c7c2 100644 --- a/README.md +++ b/README.md @@ -94,7 +94,18 @@ services: If you only need terminal access without the full JupyterLab interface, use [ttyd](https://github.com/tsl0922/ttyd) - a lightweight web-based terminal. -To add ttyd to your Linux distro, use the `features/src/ttyd` feature and configure the container command: +To add ttyd to your Linux distro, add the [ttyd feature](https://github.com/ar90n/devcontainer-features/tree/main/src/ttyd) to your `.devcontainer.json`: + +```json +// .devcontainer.json +{ + "features": { + "ghcr.io/ar90n/devcontainer-features/ttyd:1": {} + } +} +``` + +Then configure the container command in your `docker-compose.yaml`: ```yaml # docker-compose.yaml diff --git a/features/src/ttyd/README.md b/features/src/ttyd/README.md deleted file mode 100644 index bb8a3ce8..00000000 --- a/features/src/ttyd/README.md +++ /dev/null @@ -1,39 +0,0 @@ -# ttyd - Web-based Terminal - -Installs [ttyd](https://github.com/tsl0922/ttyd), a simple command-line tool for sharing your terminal over the web. - -## Usage - -```json -"features": { - "./.devcontainer/features/ttyd": { - "version": "latest", - "port": "7681" - } -} -``` - -## Options - -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| version | string | `latest` | Version of ttyd to install (e.g., '1.7.7' or 'latest') | -| port | string | `7681` | Port ttyd will listen on | - -## Running ttyd - -Configure ttyd as the container command in your `docker-compose.yaml`: - -```yaml -services: - app: - command: ["ttyd", "-p", "7681", "bash"] - ports: - - 7681:7681 -``` - -Then access the terminal in your browser at `http://localhost:7681`. - -## Example - -See the repository README for examples of using ttyd with Linux distributions that don't have a built-in UI. diff --git a/features/src/ttyd/devcontainer-feature.json b/features/src/ttyd/devcontainer-feature.json deleted file mode 100644 index f6673811..00000000 --- a/features/src/ttyd/devcontainer-feature.json +++ /dev/null @@ -1,19 +0,0 @@ -{ - "id": "ttyd", - "name": "ttyd - Web-based Terminal", - "version": "1.0.0", - "description": "Installs ttyd, a simple command-line tool for sharing terminal over the web", - "documentationURL": "https://github.com/tsl0922/ttyd", - "options": { - "version": { - "type": "string", - "default": "latest", - "description": "Version of ttyd to install (e.g., '1.7.7' or 'latest')" - }, - "port": { - "type": "string", - "default": "7681", - "description": "Port ttyd will listen on" - } - } -} diff --git a/features/src/ttyd/install.sh b/features/src/ttyd/install.sh deleted file mode 100755 index 13917af2..00000000 --- a/features/src/ttyd/install.sh +++ /dev/null @@ -1,78 +0,0 @@ -#!/bin/bash -set -o errexit -o nounset -o pipefail -o xtrace - -# ttyd feature install script -# Installs ttyd - a simple tool for sharing terminal over the web - -readonly TTYD_VERSION="${VERSION:-"latest"}" -readonly TTYD_PORT="${PORT:-"7681"}" - -echo "Installing ttyd ${TTYD_VERSION}..." - -# Check for root -if [ "$(id -u)" -ne 0 ]; then - echo 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.' - exit 1 -fi - -# Detect OS -. /etc/os-release -if [ "${ID}" = "debian" ] || [ "${ID_LIKE}" = "debian" ]; then - ADJUSTED_ID="debian" -elif [[ "${ID}" = "rhel" || "${ID}" = "fedora" || "${ID_LIKE}" = *"rhel"* || "${ID_LIKE}" = *"fedora"* ]]; then - ADJUSTED_ID="rhel" -else - echo "Linux distro ${ID} not supported." - exit 1 -fi - -# Install dependencies -if [ "${ADJUSTED_ID}" = "debian" ]; then - apt-get update - apt-get install -y --no-install-recommends curl ca-certificates -elif [ "${ADJUSTED_ID}" = "rhel" ]; then - yum install -y curl ca-certificates -fi - -# Determine architecture -ARCH="$(uname -m)" -case "${ARCH}" in - x86_64) ARCH="x86_64" ;; - aarch64) ARCH="aarch64" ;; - armv7l) ARCH="armv7" ;; - *) echo "Unsupported architecture: ${ARCH}"; exit 1 ;; -esac - -# Get latest version if needed -if [ "${TTYD_VERSION}" = "latest" ]; then - TTYD_RELEASE_VERSION=$(curl -sL https://api.github.com/repos/tsl0922/ttyd/releases/latest | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/' || echo "") - # Fallback to a known version if API call fails - if [ -z "${TTYD_RELEASE_VERSION}" ]; then - echo "Warning: Could not fetch latest version from GitHub API, using fallback version 1.7.7" - TTYD_RELEASE_VERSION="1.7.7" - fi -else - TTYD_RELEASE_VERSION="${TTYD_VERSION}" -fi - -echo "Installing ttyd version ${TTYD_RELEASE_VERSION} for ${ARCH}..." - -# Download and install ttyd -DOWNLOAD_URL="https://github.com/tsl0922/ttyd/releases/download/${TTYD_RELEASE_VERSION}/ttyd.${ARCH}" -curl -sL "${DOWNLOAD_URL}" -o /usr/local/bin/ttyd -chmod +x /usr/local/bin/ttyd - -# Verify installation -if ! ttyd --version; then - echo "Failed to install ttyd" - exit 1 -fi - -# Clean up -if [ "${ADJUSTED_ID}" = "debian" ]; then - rm -rf /var/lib/apt/lists/* -fi - -echo "ttyd ${TTYD_RELEASE_VERSION} installed successfully!" -echo "Default port: ${TTYD_PORT}" -echo "To run: ttyd -p ${TTYD_PORT} bash" diff --git a/features/test/ttyd/README.md b/features/test/ttyd/README.md deleted file mode 100644 index b60dac04..00000000 --- a/features/test/ttyd/README.md +++ /dev/null @@ -1,68 +0,0 @@ -# ttyd Feature Tests - -This directory contains tests for the `ttyd` devcontainer feature. - -## Test Structure - -Following the [devcontainer feature testing guidelines](https://github.com/devcontainers/cli/blob/main/docs/features/test.md), this directory contains: - -- **`scenarios.json`**: Defines test scenarios with different feature configurations -- **`test.sh`**: Test script that validates ttyd installation and functionality - -## Test Scenarios - -### ttyd-default -Tests ttyd installation with default options (latest version, default port 7681) - -### ttyd-specific-version -Tests ttyd installation with a specific version (1.7.7) - -### ttyd-custom-port -Tests ttyd installation with a custom port (8080) - -## Running Tests Locally - -### Prerequisites - -Install the devcontainer CLI: -```bash -npm install -g @devcontainers/cli -``` - -### Run All Tests - -From the `features` directory: -```bash -cd features -devcontainer features test --features ttyd --base-image mcr.microsoft.com/devcontainers/base:ubuntu . -``` - -### Run Specific Scenario - -```bash -cd features -devcontainer features test \ - --features ttyd \ - --base-image mcr.microsoft.com/devcontainers/base:ubuntu \ - --filter ttyd-default \ - . -``` - -**Note**: Running tests locally on macOS with Colima may encounter Docker mount issues. The tests will run properly in Linux environments and in CI. - -## What the Tests Verify - -The test script (`test.sh`) verifies: - -1. ✅ ttyd is installed and in PATH -2. ✅ ttyd version command works -3. ✅ ttyd version output matches expected format -4. ✅ ttyd help command works - -## CI/CD Integration - -These tests run automatically in GitHub Actions when: -- A pull request modifies `features/**` -- Changes are pushed to the master branch - -See `.github/workflows/test-scripts.yaml` for the CI configuration. diff --git a/features/test/ttyd/scenarios.json b/features/test/ttyd/scenarios.json deleted file mode 100644 index 6a4fa3f1..00000000 --- a/features/test/ttyd/scenarios.json +++ /dev/null @@ -1,24 +0,0 @@ -{ - "ttyd-default": { - "image": "mcr.microsoft.com/devcontainers/base:ubuntu", - "features": { - "ttyd": {} - } - }, - "ttyd-specific-version": { - "image": "mcr.microsoft.com/devcontainers/base:ubuntu", - "features": { - "ttyd": { - "version": "1.7.7" - } - } - }, - "ttyd-custom-port": { - "image": "mcr.microsoft.com/devcontainers/base:ubuntu", - "features": { - "ttyd": { - "port": "8080" - } - } - } -} diff --git a/features/test/ttyd/test.sh b/features/test/ttyd/test.sh deleted file mode 100755 index 4ca60334..00000000 --- a/features/test/ttyd/test.sh +++ /dev/null @@ -1,26 +0,0 @@ -#!/bin/bash - -# This test file will be executed against an auto-generated devcontainer.json that -# includes the 'ttyd' feature with various options. - -set -e - -# Optional: Import test library bundled with the devcontainer CLI -# See https://github.com/devcontainers/cli/blob/HEAD/docs/features/test.md#dev-container-features-test-lib -# Provides the 'check' command to execute tests and the 'reportResults' function to report results. -source dev-container-features-test-lib - -# Feature-specific tests -# The 'check' command takes a label and a command to run. - -check "ttyd installed" which ttyd - -check "ttyd version" ttyd --version - -check "ttyd executable" bash -c "ttyd --version 2>&1 | grep -E 'ttyd version [0-9]+\.[0-9]+\.[0-9]+'" - -check "ttyd help" ttyd --help - -# Report result -# If any of the checks above exited with a non-zero exit code, the test will fail. -reportResults diff --git a/scripts/create-custom-app.sh b/scripts/create-custom-app.sh index c20ad6aa..0665e371 100755 --- a/scripts/create-custom-app.sh +++ b/scripts/create-custom-app.sh @@ -38,8 +38,10 @@ fi readonly -f readonly APP_DIR="src/${APP_NAME}" -readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -readonly REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +readonly SCRIPT_DIR +REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" +readonly REPO_ROOT # Create app directory echo "Creating app directory: ${APP_DIR}" From 55c63808bf4fca7255bc1c2749ef88a61385e1db Mon Sep 17 00:00:00 2001 From: Yu Hu Date: Tue, 23 Dec 2025 12:26:28 -0500 Subject: [PATCH 04/11] clean up --- README.md | 45 ++++++++++++++++++---- scripts/create-custom-app.sh | 57 +++++++++++++-------------- scripts/test/create-custom-app.bats | 60 +++++++++++++++-------------- 3 files changed, 95 insertions(+), 67 deletions(-) diff --git a/README.md b/README.md index b219c7c2..2d2bbfdc 100644 --- a/README.md +++ b/README.md @@ -112,13 +112,26 @@ Then configure the container command in your `docker-compose.yaml`: services: app: container_name: "application-server" - image: "ubuntu:22.04" - command: ["ttyd", "-p", "7681", "bash"] + image: "mcr.microsoft.com/devcontainers/base:ubuntu" + # Container runs as root for SYS_ADMIN capabilities, but terminal runs as vscode user + command: ["ttyd", "-W", "-p", "7681", "su", "-", "vscode"] ports: - 7681:7681 + cap_add: + - SYS_ADMIN + devices: + - /dev/fuse + security_opt: + - apparmor:unconfined # ... rest of configuration ``` +**Important**: +- The `-W` flag makes the terminal writable (interactive). Without it, the terminal will be read-only. +- The container runs as root (needed for SYS_ADMIN and /dev/fuse capabilities), but the terminal session runs as the `vscode` user via `su - vscode`. + +**Tip**: Use `mcr.microsoft.com/devcontainers/base:ubuntu` instead of plain `ubuntu:22.04` - it comes with a pre-configured `vscode` user and common development tools. + #### Option 3: VS Code Server (Full IDE Experience) For a full IDE experience, use the [vscode-server feature](https://github.com/devcontainers-extra/features/tree/main/src/vscode-server) which provides VS Code in the browser with built-in terminal access. @@ -129,18 +142,36 @@ To run and debug your app locally: 1. **Install the devcontainer CLI**: Follow the installation instructions at https://code.visualstudio.com/docs/devcontainers/devcontainer-cli -2. **Build your app**: +2. **Create the Docker network**: Workbench apps require an external Docker network named `app-network` ```bash - cd src/ - devcontainer build --workspace-folder . + docker network create app-network ``` -3. **Start your app**: +3. **Comment out Workbench-specific commands**: For local testing, you should comment out the `postCreateCommand` and `postStartCommand` in your `.devcontainer.json` since these scripts are designed to run in the Workbench environment and may fail locally: + ```json + { + // "postCreateCommand": [ + // "./startupscript/post-startup.sh", + // "username", + // "/home/username", + // "gcp" + // ], + // "postStartCommand": [ + // "./startupscript/remount-on-restart.sh", + // "username", + // "/home/username", + // "gcp" + // ] + } + ``` + +4. **Run your app**: ```bash + cd src/ devcontainer up --workspace-folder . ``` -4. **Access your app**: Once the container is running, you can access it at `localhost:` where `` is the port you specified in your configuration (e.g., `localhost:8888` for Jupyter) +5. **Access your app**: Once the container is running, you can access it at `localhost:` where `` is the port you specified in your configuration (e.g., `localhost:8888` for Jupyter, `localhost:7681` for ttyd) ## How to use diff --git a/scripts/create-custom-app.sh b/scripts/create-custom-app.sh index 0665e371..b1b41d5a 100755 --- a/scripts/create-custom-app.sh +++ b/scripts/create-custom-app.sh @@ -58,14 +58,14 @@ cat > "${REPO_ROOT}/${APP_DIR}/.devcontainer.json" < "${REPO_ROOT}/${APP_DIR}/.devcontainer.json" < "${REPO_ROOT}/${APP_DIR}/devcontainer-template.json" < Date: Tue, 23 Dec 2025 12:31:36 -0500 Subject: [PATCH 05/11] clean up example --- scripts/create-custom-app.sh | 3 +- src/example/.devcontainer.json | 53 +++----------------------- src/example/README.md | 53 ++++++++++++++++---------- src/example/devcontainer-template.json | 35 +++-------------- src/example/docker-compose.yaml | 11 +++--- 5 files changed, 51 insertions(+), 104 deletions(-) diff --git a/scripts/create-custom-app.sh b/scripts/create-custom-app.sh index b1b41d5a..1d260859 100755 --- a/scripts/create-custom-app.sh +++ b/scripts/create-custom-app.sh @@ -73,8 +73,7 @@ cat > "${REPO_ROOT}/${APP_DIR}/.devcontainer.json" < Date: Tue, 23 Dec 2025 12:35:19 -0500 Subject: [PATCH 06/11] update README --- README.md | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 2d2bbfdc..a5d3aa8c 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,41 @@ https://containers.dev/ ## Developing a New App -To create a custom app for Workbench: +### Quick Start (Recommended) + +The fastest way to create a custom app is using the `create-custom-app.sh` script: + +```bash +./scripts/create-custom-app.sh [username] [home-dir] +``` + +**Example** (this created the current example app): +```bash +./scripts/create-custom-app.sh example quay.io/jupyter/base-notebook 8888 jovyan /home/jovyan +``` + +This script generates a complete app structure in `src//` with: +- `.devcontainer.json` - Devcontainer configuration +- `docker-compose.yaml` - Docker Compose setup with ttyd terminal +- `devcontainer-template.json` - Template metadata +- `README.md` - App-specific documentation + +**Arguments:** +- `app-name`: Name of your custom app (e.g., `my-jupyter-app`) +- `docker-image`: Docker image to use (e.g., `jupyter/base-notebook`, `rocker/rstudio`) +- `port`: Port your app exposes (e.g., `8888` for Jupyter, `8787` for RStudio) +- `username`: (Optional) User inside container (default: `root`) +- `home-dir`: (Optional) Home directory (default: `/root` or `/home/`) + +After running the script: +1. Review and customize the generated files in `src//` +2. Test your app: `cd test && ./test.sh ` +3. Commit and push to your forked repository +4. Create a custom app in Workbench UI using your repository + +### Manual Setup (Advanced) + +If you need more control, you can manually create a custom app: 1. **Fork this repository** to your own GitHub account or organization @@ -54,8 +88,6 @@ To create a custom app for Workbench: - Include any needed features from `features/src/` (e.g., `workbench-tools`) - Use template option `${templateOption:cloud}` to specify the cloud provider (GCP or AWS) - You can use the script `./scripts/create-custom-app.sh` to generate a basic devcontainer structure from these parameters. - 5. **Test your app**: - Run the test script: `cd test && ./test.sh ` - Create a custom app in Workbench UI pointing to your forked repo and branch From 7ab1f92f46300142a49c11d67ea96df2a2ea1271 Mon Sep 17 00:00:00 2001 From: Yu Hu Date: Tue, 23 Dec 2025 12:43:32 -0500 Subject: [PATCH 07/11] clean up ttyd --- scripts/create-custom-app.sh | 3 --- src/example/docker-compose.yaml | 3 --- 2 files changed, 6 deletions(-) diff --git a/scripts/create-custom-app.sh b/scripts/create-custom-app.sh index 1d260859..a3967da9 100755 --- a/scripts/create-custom-app.sh +++ b/scripts/create-custom-app.sh @@ -98,9 +98,6 @@ services: # Workbench UI. ports: - ${PORT}:${PORT} - # Start ttyd with writable mode (-W flag) for interactive terminal - # Container runs as root for capabilities, but terminal runs as configured user - command: ["ttyd", "-W", "-p", "${PORT}", "su", "-", "${USERNAME}"] # The service must be connected to the "app-network" Docker network networks: - app-network diff --git a/src/example/docker-compose.yaml b/src/example/docker-compose.yaml index cc3224f4..d5975834 100644 --- a/src/example/docker-compose.yaml +++ b/src/example/docker-compose.yaml @@ -14,9 +14,6 @@ services: # Workbench UI. ports: - 8888:8888 - # Start ttyd with writable mode (-W flag) for interactive terminal - # Container runs as root for capabilities, but terminal runs as configured user - command: ["ttyd", "-W", "-p", "8888", "su", "-", "jovyan"] # The service must be connected to the "app-network" Docker network networks: - app-network From daf91e3a5810f2e09db79540ae16678f2063ef56 Mon Sep 17 00:00:00 2001 From: Yu Hu Date: Tue, 23 Dec 2025 13:30:09 -0500 Subject: [PATCH 08/11] add an example for ubuntu image with ttyd --- scripts/test/create-custom-app.bats | 2 - src/ubuntu-example/.devcontainer.json | 28 ++++++++ src/ubuntu-example/README.md | 67 +++++++++++++++++++ src/ubuntu-example/devcontainer-template.json | 14 ++++ src/ubuntu-example/docker-compose.yaml | 38 +++++++++++ 5 files changed, 147 insertions(+), 2 deletions(-) create mode 100644 src/ubuntu-example/.devcontainer.json create mode 100644 src/ubuntu-example/README.md create mode 100644 src/ubuntu-example/devcontainer-template.json create mode 100644 src/ubuntu-example/docker-compose.yaml diff --git a/scripts/test/create-custom-app.bats b/scripts/test/create-custom-app.bats index 917a3a0e..d21b63bf 100755 --- a/scripts/test/create-custom-app.bats +++ b/scripts/test/create-custom-app.bats @@ -84,8 +84,6 @@ teardown() { grep -q 'image: "python:3.11"' "src/test-app/docker-compose.yaml" grep -q '8080:8080' "src/test-app/docker-compose.yaml" grep -q 'work:/home/testuser/work' "src/test-app/docker-compose.yaml" - grep -q '"8080"' "src/test-app/docker-compose.yaml" - grep -q '"testuser"' "src/test-app/docker-compose.yaml" # Check for required workbench settings grep -q 'container_name: "application-server"' "src/test-app/docker-compose.yaml" diff --git a/src/ubuntu-example/.devcontainer.json b/src/ubuntu-example/.devcontainer.json new file mode 100644 index 00000000..2d31e369 --- /dev/null +++ b/src/ubuntu-example/.devcontainer.json @@ -0,0 +1,28 @@ +{ + "name": "ubuntu-example", + "dockerComposeFile": "docker-compose.yaml", + "service": "app", + "shutdownAction": "none", + "workspaceFolder": "/workspace", + "postCreateCommand": [ + "./startupscript/post-startup.sh", + "vscode", + "/home/vscode", + "${templateOption:cloud}" + ], + "postStartCommand": [ + "./startupscript/remount-on-restart.sh", + "vscode", + "/home/vscode", + "${templateOption:cloud}" + ], + "features": { + "ghcr.io/devcontainers/features/java:1": { + "version": "17" + }, + "ghcr.io/devcontainers/features/aws-cli:1": {}, + "ghcr.io/dhoeric/features/google-cloud-cli:1": {}, + "ghcr.io/ar90n/devcontainer-features/ttyd:1": {} + }, + "remoteUser": "root" +} diff --git a/src/ubuntu-example/README.md b/src/ubuntu-example/README.md new file mode 100644 index 00000000..e4b5aaa8 --- /dev/null +++ b/src/ubuntu-example/README.md @@ -0,0 +1,67 @@ +# ubuntu-example + +Custom Workbench application based on mcr.microsoft.com/devcontainers/base:ubuntu. + +## Configuration + +- **Image**: mcr.microsoft.com/devcontainers/base:ubuntu +- **Port**: 7681 +- **User**: vscode +- **Home Directory**: /home/vscode + +## Access + +This app uses [ttyd](https://github.com/tsl0922/ttyd) to provide web-based terminal access. + +Once deployed in Workbench, access your terminal at the app URL (port 7681). + +For local testing: +1. Create Docker network: `docker network create app-network` +2. Run the app: `devcontainer up --workspace-folder .` +3. Access at: `http://localhost:7681` + +## Development + +This app was created as a reference implementation for building custom distro image apps: + +1. **Initial scaffold** - Created using the `scripts/create-custom-app.sh` script: + ```bash + ./scripts/create-custom-app.sh ubuntu-example mcr.microsoft.com/devcontainers/base:ubuntu 7681 vscode /home/vscode + ``` + +2. **Added ttyd feature** - Modified `.devcontainer.json` to include the ttyd devcontainer feature: + ```json + "ghcr.io/ar90n/devcontainer-features/ttyd:1": {} + ``` + +3. **Configured user** - Added `user: vscode` in `docker-compose.yaml` (line 13) because the default user is root, but we want to run as the vscode user for better permissions handling. + +4. **Added ttyd command** - Added the ttyd startup command in `docker-compose.yaml` (line 14): + ```yaml + command: ["ttyd", "-W", "-p", "7681", "bash"] + ``` + This starts ttyd with web terminal access on port 7681. Since we're already running as the vscode user (via `user: vscode` on line 13), we can start bash directly. + +## Customization + +Edit the following files to customize your app: + +- `.devcontainer.json` - Devcontainer configuration and features +- `docker-compose.yaml` - Docker Compose configuration (change the `command` to customize ttyd options) +- `devcontainer-template.json` - Template options and metadata + +## Testing + +To test this app template: + +```bash +cd test +./test.sh ubuntu-example +``` + +## Usage + +1. Fork the repository +2. Modify the configuration files as needed +3. In Workbench UI, create a custom app pointing to your forked repository +4. Select this app template (ubuntu-example) diff --git a/src/ubuntu-example/devcontainer-template.json b/src/ubuntu-example/devcontainer-template.json new file mode 100644 index 00000000..82910068 --- /dev/null +++ b/src/ubuntu-example/devcontainer-template.json @@ -0,0 +1,14 @@ +{ + "id": "ubuntu-example", + "version": "1.0.0", + "name": "ubuntu-example", + "description": "Custom Workbench app: ubuntu-example (Image: mcr.microsoft.com/devcontainers/base:ubuntu, Port: 7681, User: vscode)", + "options": { + "cloud": { + "type": "string", + "enum": ["gcp", "aws"], + "default": "gcp", + "description": "Cloud provider (gcp or aws)" + } + } +} diff --git a/src/ubuntu-example/docker-compose.yaml b/src/ubuntu-example/docker-compose.yaml new file mode 100644 index 00000000..10d7ecab --- /dev/null +++ b/src/ubuntu-example/docker-compose.yaml @@ -0,0 +1,38 @@ +services: + app: + # The container name must be "application-server" + container_name: "application-server" + # This can be either a pre-existing image or built from a Dockerfile + image: "mcr.microsoft.com/devcontainers/base:ubuntu" + # build: + # context: . + restart: always + volumes: + - .:/workspace:cached + - work:/home/vscode/work + user: vscode + command: ["ttyd", "-W", "-p", "7681", "bash"] + # The port specified here will be forwarded and accessible from the + # Workbench UI. + ports: + - 7681:7681 + # The service must be connected to the "app-network" Docker network + networks: + - app-network + # SYS_ADMIN and fuse are required to mount workspace resources into the + # container. + cap_add: + - SYS_ADMIN + devices: + - /dev/fuse + security_opt: + - apparmor:unconfined + +volumes: + work: + +networks: + # The Docker network must be named "app-network". This is an external network + # that is created outside of this docker-compose file. + app-network: + external: true From f32394cd1a04ac94f06a861884dd308156463f90 Mon Sep 17 00:00:00 2001 From: Yu Hu Date: Tue, 23 Dec 2025 14:31:33 -0500 Subject: [PATCH 09/11] address comment --- .github/workflows/test-pr.yaml | 6 +++++- README.md | 19 +++++++++++++++++++ scripts/create-custom-app.sh | 2 -- src/example/README.md | 2 -- 4 files changed, 24 insertions(+), 5 deletions(-) diff --git a/.github/workflows/test-pr.yaml b/.github/workflows/test-pr.yaml index e48c6064..1fd1ea4d 100644 --- a/.github/workflows/test-pr.yaml +++ b/.github/workflows/test-pr.yaml @@ -25,7 +25,9 @@ jobs: - 'startupscript/**' - 'test/**' config: - - jupyter-template: + - example: + user: jovyan + jupyter-template: user: jupyter r-analysis: user: rstudio @@ -71,6 +73,8 @@ jobs: filters: - 'src/aou-common/**' - 'src/workbench-jupyter-parabricks/**' + ubuntu-example: + user: vscode outputs: apps: ${{ steps.output.outputs.apps }} steps: diff --git a/README.md b/README.md index a5d3aa8c..4970a412 100644 --- a/README.md +++ b/README.md @@ -44,6 +44,25 @@ This script generates a complete app structure in `src//` with: - `devcontainer-template.json` - Template metadata - `README.md` - App-specific documentation +**Using a Dockerfile instead of a Docker image:** + +If you don't have a pre-built Docker image and only have a Dockerfile: + +1. Run the script with an empty image parameter: + ```bash + ./scripts/create-custom-app.sh my-app "" 8888 myuser /home/myuser + ``` + +2. In the generated `src/my-app/docker-compose.yaml`, uncomment the `build` section: + ```yaml + build: + context: . + ``` + +3. Add your `Dockerfile` to `src/my-app/` + +4. Remove the `image:` line from the `docker-compose.yaml` + **Arguments:** - `app-name`: Name of your custom app (e.g., `my-jupyter-app`) - `docker-image`: Docker image to use (e.g., `jupyter/base-notebook`, `rocker/rstudio`) diff --git a/scripts/create-custom-app.sh b/scripts/create-custom-app.sh index a3967da9..68f7be16 100755 --- a/scripts/create-custom-app.sh +++ b/scripts/create-custom-app.sh @@ -155,8 +155,6 @@ Custom Workbench application based on ${DOCKER_IMAGE}. ## Access -This app uses [ttyd](https://github.com/tsl0922/ttyd) to provide web-based terminal access. - Once deployed in Workbench, access your terminal at the app URL (port ${PORT}). For local testing: diff --git a/src/example/README.md b/src/example/README.md index 3b16b808..429d4844 100644 --- a/src/example/README.md +++ b/src/example/README.md @@ -11,8 +11,6 @@ Custom Workbench application based on quay.io/jupyter/base-notebook. ## Access -This app uses [ttyd](https://github.com/tsl0922/ttyd) to provide web-based terminal access. - Once deployed in Workbench, access your terminal at the app URL (port 8888). For local testing: From 0a542e0e9e0995c5565c9ff6260109483b7048e2 Mon Sep 17 00:00:00 2001 From: Yu Hu Date: Tue, 23 Dec 2025 14:35:03 -0500 Subject: [PATCH 10/11] udpate --- README.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 4970a412..1a552012 100644 --- a/README.md +++ b/README.md @@ -164,8 +164,8 @@ services: app: container_name: "application-server" image: "mcr.microsoft.com/devcontainers/base:ubuntu" - # Container runs as root for SYS_ADMIN capabilities, but terminal runs as vscode user - command: ["ttyd", "-W", "-p", "7681", "su", "-", "vscode"] + user: vscode + command: ["ttyd", "-W", "-p", "7681", "bash"] ports: - 7681:7681 cap_add: @@ -179,9 +179,6 @@ services: **Important**: - The `-W` flag makes the terminal writable (interactive). Without it, the terminal will be read-only. -- The container runs as root (needed for SYS_ADMIN and /dev/fuse capabilities), but the terminal session runs as the `vscode` user via `su - vscode`. - -**Tip**: Use `mcr.microsoft.com/devcontainers/base:ubuntu` instead of plain `ubuntu:22.04` - it comes with a pre-configured `vscode` user and common development tools. #### Option 3: VS Code Server (Full IDE Experience) From 5686f7c92f326dd84a1fb3e0d66747b25b7e1722 Mon Sep 17 00:00:00 2001 From: Yu Hu Date: Tue, 23 Dec 2025 14:46:13 -0500 Subject: [PATCH 11/11] fix smoke test --- scripts/create-custom-app.sh | 12 ++++++++++-- src/example/.devcontainer.json | 6 ++++-- src/example/devcontainer-template.json | 6 ++++++ src/ubuntu-example/.devcontainer.json | 6 ++++-- src/ubuntu-example/devcontainer-template.json | 6 ++++++ 5 files changed, 30 insertions(+), 6 deletions(-) diff --git a/scripts/create-custom-app.sh b/scripts/create-custom-app.sh index 68f7be16..0ea97314 100755 --- a/scripts/create-custom-app.sh +++ b/scripts/create-custom-app.sh @@ -60,13 +60,15 @@ cat > "${REPO_ROOT}/${APP_DIR}/.devcontainer.json" < "${REPO_ROOT}/${APP_DIR}/devcontainer-template.json" <