Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions .env

This file was deleted.

21 changes: 21 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# .env.example - Optional environment file for AltarDocker
#
# docker-compose.yml has sensible defaults built-in!
# You only need this .env file if you want to customize settings.
#
# To use: Copy this file to .env and modify the values below
# Without .env: docker-compose.yml uses defaults (sacred, 27017, minio_admin, etc.)

# MongoDB Configuration
MONGO_DB=sacred
MONGO_PORT=27017

# MinIO Configuration (S3-compatible storage)
MINIO_ROOT_USER=minio_admin
MINIO_ROOT_PASSWORD=changeme123
MINIO_S3_PORT=9000
MINIO_CONSOLE_PORT=9001

# Volume Paths (where data is stored)
MONGO_DATA_PATH=./data/mongo/
MINIO_DATA_PATH=./data/minio/
53 changes: 53 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Validate Docker Compose

on:
push:
branches: [ main, no_credentials ]
pull_request:
branches: [ main ]

jobs:
validate-compose:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Validate docker-compose.yml syntax
run: |
docker compose -f docker-compose.yml config > /dev/null
echo "✅ docker-compose.yml is valid"

- name: Check .env.example has all required variables
run: |
echo "Checking required variables in .env.example..."
grep -q "MONGO_DB=" .env.example || (echo "❌ Missing MONGO_DB" && exit 1)
grep -q "MONGO_PORT=" .env.example || (echo "❌ Missing MONGO_PORT" && exit 1)
grep -q "MINIO_ROOT_USER=" .env.example || (echo "❌ Missing MINIO_ROOT_USER" && exit 1)
grep -q "MINIO_ROOT_PASSWORD=" .env.example || (echo "❌ Missing MINIO_ROOT_PASSWORD" && exit 1)
grep -q "MONGO_DATA_PATH=" .env.example || (echo "❌ Missing MONGO_DATA_PATH" && exit 1)
grep -q "MINIO_DATA_PATH=" .env.example || (echo "❌ Missing MINIO_DATA_PATH" && exit 1)
echo "✅ All required variables present"

- name: Test docker-compose.yml with defaults (no .env)
run: |
# Test without .env file to verify defaults work
docker compose -f docker-compose.yml up --dry-run mongo
echo "✅ docker-compose.yml works with defaults (no .env)"

- name: Test docker-compose.yml with .env overrides
run: |
cp .env.example .env
# Test all services
docker compose -f docker-compose.yml up --dry-run
echo "✅ docker-compose.yml all services with .env valid"

- name: Pull Docker images to verify they exist
run: |
docker pull mongo:6
docker pull quay.io/minio/minio:latest
echo "✅ All Docker images are accessible"
228 changes: 45 additions & 183 deletions DEPLOY.md
Original file line number Diff line number Diff line change
@@ -1,219 +1,81 @@
# AltarDocker — Local Deployment Guide
# AltarDocker — Deployment Guide

This guide helps you start a minimal stack locally using Docker Compose:
Complete guide for deploying MongoDB and MinIO infrastructure for Sacred experiment tracking.

**Services included:**
- MongoDB for experiment metadata
- MinIO (S3-compatible) for raw data (optional)
- Omniboard connected to MongoDB
- AltarExtractor for browsing Sacred experiments (optional)
- MinIO for S3-compatible object storage

> **Note:** Omniboard is managed by [AltarViewer](../AltarViewer), AltarExtractor is [deployed separately](../AltarExtractor).

---

## Prerequisites
- Docker >= 24 and Docker Compose v2 installed (https://docs.docker.com/engine/install/)
- At least 2 GB of free disk space (more depending on the size of your data)
- Docker >= 24 and Docker Compose v2 installed
- At least 2 GB free disk space

---

## Project Layout
```
AltarDocker/
├─ .env # Environment variables (create this)
├─ docker-compose.yml # Docker Compose configuration
├─ DEPLOY.md # This file
├─ MANAGE_USERS.md # User management guide
└─ mongo_dump.sh # Backup script example
```
## 1) (Optional) Customize Settings

---
**The compose file works without a `.env` file!** It has built-in defaults:
- MongoDB: `sacred` database on port `27017`
- MinIO: `minio_admin` / `changeme123` on ports `9000` and `9001`
- Data stored in `./data/mongo/` and `./data/minio/`

**To customize**, copy `.env.example` to `.env` and modify the values:

## 1) Create the `.env` file
```bash
cp .env.example .env
# Edit .env with your preferred values
```

Create a `.env` file in this folder with strong passwords:
The `.env.example` file contains:

```dotenv
# MongoDB (root account for initial setup)
MONGO_ROOT_USER=admin
MONGO_ROOT_PASSWORD=change_me_mongo_password
# MongoDB Configuration
MONGO_DB=sacred
MONGO_PORT=27017

# MinIO (S3 + console)
# MinIO Configuration
MINIO_ROOT_USER=minio_admin
MINIO_ROOT_PASSWORD=change_me_minio_password

# Host port for Omniboard (internal port is 9000)
OMNIBOARD_HOST_PORT=9004
MINIO_ROOT_PASSWORD=your_secure_password
MINIO_S3_PORT=9000
MINIO_CONSOLE_PORT=9001

# Host port for AltarExtractor (internal port is 8050)
EXTRACTOR_HOST_PORT=8050
# Volume Paths (where data is stored)
MONGO_DATA_PATH=./data/mongo/
MINIO_DATA_PATH=./data/minio/
```

---

## 2) Edit `docker-compose.yml`

The data for MinIO and MongoDB will be saved in the folders specified in `volumes` (outside the Docker container). Update the volume paths to your preferred locations:

```yaml
services:
mongo:
image: mongo:6
container_name: mongo
restart: unless-stopped
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_ROOT_USER}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_ROOT_PASSWORD}
MONGO_INITDB_DATABASE: ${MONGO_DB}
ports:
- "27017:27017"
volumes:
- /path/to/your/mongo_data:/data/db # Change this path

minio:
image: quay.io/minio/minio:latest
container_name: minio
restart: unless-stopped
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
ports:
- "9000:9000" # S3 API
- "9001:9001" # Web console
volumes:
- /path/to/your/minio_data:/data # Change this path
profiles:
- minio

omniboard:
image: vivekratnavel/omniboard:latest
container_name: omniboard
restart: unless-stopped
depends_on:
- mongo
ports:
- "${OMNIBOARD_HOST_PORT}:9000"
command: [
"--mu",
"mongodb://${MONGO_ROOT_USER}:${MONGO_ROOT_PASSWORD}@mongo:27017/?authSource=admin&authMechanism=SCRAM-SHA-1",
"${MONGO_DB}"
]

extractor:
build:
context: ../AltarExtractor
dockerfile: Dockerfile
container_name: altar-extractor
restart: unless-stopped
depends_on:
- mongo
ports:
- "${EXTRACTOR_HOST_PORT:-8050}:8050"
environment:
- PORT=8050
- SACRED_DB_NAME=${MONGO_DB}
profiles:
- extractor

volumes:
mongo_data:
minio_data:
```

---
## 2) Start the Services

## 3) Start the Docker containers

### Basic stack (MongoDB + Omniboard)
```bash
docker compose up -d
docker ps
```

### With MinIO
```bash
docker compose --profile minio up -d
docker ps
```

### With AltarExtractor
```bash
docker compose --profile extractor up -d
docker ps
```

### With both MinIO and AltarExtractor
```bash
docker compose --profile minio --profile extractor up -d
docker ps
```

> **Note:** On Linux, you may need to prefix commands with `sudo`.

### URLs
**All services start automatically:**
- MongoDB on port 27017
- MinIO S3 API on port 9000
- MinIO Console on port 9001

### Access URLs

| Service | URL |
|----------------|----------------------------------------------------------|
| MongoDB | `mongodb://localhost:27017` (authenticate with root) |
| MongoDB | `mongodb://localhost:27017` |
| MinIO S3 API | http://localhost:9000 |
| MinIO Console | http://localhost:9001 |
| Omniboard | http://localhost:9004 (or your `OMNIBOARD_HOST_PORT`) |
| AltarExtractor | http://localhost:8050 (or your `EXTRACTOR_HOST_PORT`) |

---

## 4) Connecting AltarExtractor to MongoDB

When you open AltarExtractor in your browser, you need to configure the MongoDB connection:

1. Open http://localhost:8050 (or your configured `EXTRACTOR_HOST_PORT`)
2. In the "Database credentials" section, enter:
- **Host**: `mongo` (the Docker service name, not localhost)
- **Port**: `27017`
- **Username**: your `MONGO_ROOT_USER` value
- **Password**: your `MONGO_ROOT_PASSWORD` value
- **Auth source**: `admin`
3. Enter your database name (e.g., `sacred`)
4. Click "Connect"

> **Tip:** Check "Save credentials" to remember your connection settings in browser local storage.

---

## 5) (Recommended) Create an app-specific MongoDB user

```bash
docker exec -it mongo mongosh -u admin -p your_password --authenticationDatabase admin
```

Inside mongosh:
```javascript
use sacred
db.createUser({
user: "sacred_rw",
pwd: "change_me",
roles: [ { role: "readWrite", db: "sacred" } ]
})
```

Then update Omniboard to use this user (edit `docker-compose.yml`):
```yaml
command: [
"--mu",
"mongodb://sacred_rw:change_me@mongo:27017/?authSource=sacred&authMechanism=SCRAM-SHA-1",
"sacred"
]
```

Restart Omniboard:
```bash
docker compose up -d omniboard
```

> **Note:** On Linux, prefix with `sudo` if needed.

---

## 6) Backups (Linux)
## Backups (Linux)

### MongoDB backup script

Expand All @@ -222,7 +84,7 @@ Edit the `mongo_dump.sh` file:
#!/bin/bash

# Dumps root repository
BASE_DIR="/home/user/mongodumps/dump" # Change to your backup location
BAS5_DIR="/home/user/mongodumps/dump" # Change to your backup location

# Timestamp (e.g., 2025-07-07)
DATE_STR=$(date +%Y-%m-%d)
Expand Down Expand Up @@ -256,7 +118,7 @@ Add:
0 2 * * * bash /path/to/mongo_dump.sh >> /path/to/cron.log 2>&1
```

---
---6

## 7) Cleanup

Expand All @@ -270,14 +132,14 @@ docker compose down

## Troubleshooting

- **Omniboard cannot connect?** Check the `--mu` URI (authSource, host, port) and that MongoDB is reachable.
- **Port conflict?** Change `OMNIBOARD_HOST_PORT`, `EXTRACTOR_HOST_PORT`, or MinIO mappings in `docker-compose.yml`.
- **Omniboard cannot connect?** Use [AltarViewer](../AltarViewer) to launch Omniboard properly configured for your database.
- **Port conflict?** Change `MONGO_PORT` or MinIO port mappings in `docker-compose.yml`.
- **S3 SDKs:** Use endpoint `http://localhost:9000` with MinIO credentials.
- **AltarExtractor can't connect?** Make sure to use `mongo` as the host (Docker service name), not `localhost`.

---

## Related

- [AltarExtractor](https://github.com/DreamRepo/AltarExtractor) — Browse and filter Sacred experiments in a web UI
- [AltarExtractor](https://github.com/DreamRepo/AltarExtractor) — Browse and filter Sacred experiments (standalone deployment)
- [AltarSender](https://github.com/DreamRepo/AltarSender) — GUI to send experiments to Sacred and MinIO
- [AltarViewer](https://github.com/DreamRepo/AltarViewer) — Launch Omniboard configured for your database
Loading