diff --git a/simulator/README.md b/simulator/README.md deleted file mode 100644 index 21a7b55..0000000 --- a/simulator/README.md +++ /dev/null @@ -1,282 +0,0 @@ -# Magic Proxy Test Environment - -## Overview - -This directory contains a **Docker-in-Docker (DinD) simulator** that allows you to safely test potentially destructive Docker operations without affecting your host system or production environment. - -The simulator is an isolated Docker environment that runs inside a container, complete with its own Docker daemon. You can run the entire magic-proxy application stack (magic-proxy + traefik) inside this isolated environment. - -## Why This Exists - -Docker operations like removing containers, pruning images, or dynamically managing the container ecosystem can be risky to test in development. This simulator provides a safe sandbox where you can: - -- Test Docker API operations in isolation -- Verify container management workflows -- Experiment with potentially destructive operations -- Debug issues without affecting other services -- Run integration tests with real Docker operations - -## Architecture - -``` -Your Host -├── docker (host Docker daemon) -└── simulator container (DinD) - ├── docker daemon (inner) - ├── magic-proxy container - └── traefik container -``` - -## Quick Start - -### One-Command Setup - -```bash -cd test-env -./up.sh -``` - -This will: -1. Build the magic-proxy image from the project root -2. Export it as a gzip tarball -3. Start the simulator with docker-compose -4. Load the pre-built image into the inner Docker daemon -5. Start magic-proxy and traefik containers inside the simulator - -### Access the Environment - -```bash -# Open a shell inside the simulator -./shell.sh - -# View logs from magic-proxy -./logs.sh -f - -# Test the API -./test.sh - -# Stop everything -./down.sh -``` - -## Files and Scripts - -### Core Files - -- **`docker-compose.yml`** - Defines the outer simulator container (DinD) -- **`filesystem/test-env.yml`** - Docker Compose file that runs INSIDE the simulator -- **`filesystem/magic-proxy.tar.gz`** - Pre-built magic-proxy Docker image (created by `prepare-simulator.sh`) - -### Helper Scripts - -| Script | Purpose | -|--------|---------| -| `prepare-simulator.sh` | Builds the magic-proxy image and exports it as gzip | -| `up.sh` | One-command startup: prepare image + start simulator | -| `down.sh` | Stop and remove the simulator container | -| `shell.sh` | Open a bash/sh shell inside the simulator | -| `logs.sh` | View logs from the inner magic-proxy container | -| `test.sh` | Test connectivity to the magic-proxy API | - -## Running Services Inside the Simulator - -### Magic Proxy - -- **Container Name**: `workspace-magic-proxy-1` -- **Port**: 3000 -- **Role**: Docker management API -- **Environment**: - - `PROXY_TYPE`: traefik - - `PROXY_OUTPUT_FILE`: traefik - -### Traefik - -- **Container Name**: `traefik` -- **Port**: 80 (mapped to host 8080 inside simulator) -- **Role**: Reverse proxy and load balancer - -## Common Workflows - -### Monitor the Application - -```bash -# Follow logs in real-time -./logs.sh -f - -# Check container status -./shell.sh -# Inside: docker ps -``` - -### Test the API - -```bash -# Automated test -./test.sh - -# Manual curl from inside simulator -./shell.sh -# Inside: curl -v http://workspace-magic-proxy-1:3000/ -``` - -### Execute Commands Inside - -```bash -./shell.sh -# Inside: -# - docker ps (list containers) -# - docker logs (view logs) -# - docker exec sh (enter container) -# - curl localhost:3000 (test the API) -``` - -### Clean Up - -```bash -# Stop everything and remove containers -./down.sh - -# Remove the pre-built image archive (to rebuild) -rm filesystem/magic-proxy.tar.gz -``` - -## Rebuilding the Image - -The `prepare-simulator.sh` script: -1. Builds the magic-proxy Docker image from `../Dockerfile` -2. Exports it as `filesystem/magic-proxy.tar.gz` -3. Shows the file size - -Run this after making code changes to magic-proxy: - -```bash -./prepare-simulator.sh -# Then reload inside the simulator: -./shell.sh -# Inside: docker compose -f /workspace/test-env.yml restart -``` - -Or use the all-in-one command: - -```bash -./up.sh # Rebuilds image and restarts everything -``` - -## Troubleshooting - -### Container Keeps Restarting - -Check the logs: -```bash -./logs.sh -``` - -Common issues: -- Missing config file at `/var/config/magic-proxy.yml` - This is expected (non-fatal initialization error) -- Port 3000 already in use - Try `./down.sh` first - -### Can't Connect to the API - -Verify containers are running: -```bash -./shell.sh -# Inside: docker ps -``` - -Test from inside the simulator: -```bash -./test.sh -``` - -### Simulator Won't Start - -Ensure: -- Docker is running on your host -- No port conflicts (8080 for traefik inside simulator) -- Sufficient disk space for the image (411MB) - -### Start Fresh - -```bash -./down.sh -rm filesystem/magic-proxy.tar.gz -./up.sh -``` - -## Architecture Notes - -### Why Docker-in-Docker? - -- **Isolation**: Operations inside don't affect the host or other containers -- **Realism**: The inner environment is a real Docker daemon with real container lifecycle -- **Safety**: Destructive operations are contained within the simulator -- **Testing**: Integration tests can use real Docker operations - -### Image Caching - -The pre-built image is cached as `filesystem/magic-proxy.tar.gz` to speed up subsequent starts. This file: -- Is created by `prepare-simulator.sh` -- Is mounted into the simulator at `/images/magic-proxy.tar.gz` -- Is automatically loaded into the inner Docker daemon on startup - -### Network - -- The simulator container runs with `privileged: true` to allow full Docker-in-Docker functionality -- The inner environment creates its own Docker network (`workspace_default`) -- Containers inside communicate via container names (DNS resolution) - -## Environment Variables - -### Magic Proxy Container - -- `PROXY_TYPE=traefik` - Use traefik backend -- `PROXY_OUTPUT_FILE=traefik` - Configuration file output name - -Configure these in `filesystem/test-env.yml` under the `magic-proxy` service. - -## Advanced Usage - -### Direct Docker Commands Inside Simulator - -```bash -docker exec simulator docker ps -docker exec simulator docker logs workspace-magic-proxy-1 -docker exec simulator docker exec workspace-magic-proxy-1 sh -``` - -### Inspect the Simulator Environment - -```bash -# List all files/mounts -docker exec simulator ls -la /workspace -docker exec simulator ls -la /images - -# Check Docker version inside -docker exec simulator docker version -``` - -### Restart Services Without Rebuilding - -```bash -docker exec simulator docker compose -f /workspace/test-env.yml restart -``` - -## Cleanup on Host - -After development, free up space: - -```bash -cd test-env -./down.sh - -# Optional: Remove the image archive to force rebuild next time -rm filesystem/magic-proxy.tar.gz -``` - -The simulator image itself (docker:27-dind) will remain in your Docker daemon unless you explicitly prune it. - -## See Also - -- `../Dockerfile` - The magic-proxy application Docker image definition -- `../package.json` - Build configuration and dependencies -- `./filesystem/test-env.yml` - Inner compose configuration diff --git a/simulator/down.sh b/simulator/down.sh deleted file mode 100755 index 2e04e03..0000000 --- a/simulator/down.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/bin/bash -# -# down.sh -# Stop and remove the simulator container and network. -# - -set -e - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" - -echo "=== Stopping Test Environment ===" -cd "$SCRIPT_DIR" - -docker compose down --remove-orphans - -echo "=== Test Environment Stopped ===" diff --git a/simulator/filesystem/.gitignore b/simulator/filesystem/.gitignore deleted file mode 100644 index c32b546..0000000 --- a/simulator/filesystem/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.tar.gz \ No newline at end of file diff --git a/simulator/filesystem/test-env.yml b/simulator/filesystem/test-env.yml deleted file mode 100644 index ad63887..0000000 --- a/simulator/filesystem/test-env.yml +++ /dev/null @@ -1,29 +0,0 @@ -version: '3.8' - -# This compose file runs INSIDE the docker-in-docker simulator. -# The magic-proxy image is pre-loaded via prepare-simulator.sh - -services: - magic-proxy: - image: magic-proxy:latest - environment: - PROXY_TYPE: "traefik" - PROXY_OUTPUT_FILE: "traefik" - volumes: - - "/var/run/docker.sock:/var/run/docker.sock" - - "/:/host:ro" # required to read the compose files - - "./config:/var/config:ro" # config directory - - "./generated:/var/generated" # output directory - restart: unless-stopped - traefik: - image: "traefik:v3.4" - container_name: "traefik" - restart: unless-stopped - security_opt: - - no-new-privileges:true - command: - - "--entryPoints.web.address=:80" - ports: - - "80:80" - volumes: - - "/var/run/docker.sock:/var/run/docker.sock:ro" \ No newline at end of file diff --git a/simulator/logs.sh b/simulator/logs.sh deleted file mode 100755 index 7444c2a..0000000 --- a/simulator/logs.sh +++ /dev/null @@ -1,13 +0,0 @@ -#!/bin/bash -# -# logs.sh -# View logs from the magic-proxy container running inside the simulator. -# Optionally follow logs if --follow or -f is passed. -# - -FOLLOW="" -if [[ "$1" == "--follow" || "$1" == "-f" ]]; then - FOLLOW="-f" -fi - -docker exec simulator docker logs $FOLLOW workspace-magic-proxy-1 diff --git a/simulator/prepare-simulator.sh b/simulator/prepare-simulator.sh deleted file mode 100755 index 775f8d7..0000000 --- a/simulator/prepare-simulator.sh +++ /dev/null @@ -1,40 +0,0 @@ -#!/bin/bash -# -# prepare-simulator.sh -# Builds the magic-proxy Docker image and exports it as a gzip tarball -# for use in the docker-in-docker simulator environment. -# - -set -e - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" -OUTPUT_DIR="$SCRIPT_DIR/filesystem" -OUTPUT_FILE="$OUTPUT_DIR/magic-proxy.tar.gz" - -echo "=== Magic Proxy Simulator Preparation ===" -echo "Project root: $PROJECT_ROOT" -echo "Output file: $OUTPUT_FILE" -echo "" - -# Ensure output directory exists -mkdir -p "$OUTPUT_DIR" - -# Build the Docker image for current platform -echo "Building magic-proxy Docker image..." -cd "$PROJECT_ROOT" -docker buildx build --platform linux/amd64,linux/arm64 -t magic-proxy:latest --load . - -# Export the image as a gzip tarball -echo "" -echo "Exporting image to $OUTPUT_FILE..." -docker save magic-proxy:latest | gzip > "$OUTPUT_FILE" - -# Show the result -SIZE=$(du -h "$OUTPUT_FILE" | cut -f1) -echo "" -echo "=== Done ===" -echo "Image exported: $OUTPUT_FILE ($SIZE)" -echo "" -echo "You can now start the simulator with:" -echo " cd $SCRIPT_DIR && docker compose up" diff --git a/simulator/shell.sh b/simulator/shell.sh deleted file mode 100755 index 0067be3..0000000 --- a/simulator/shell.sh +++ /dev/null @@ -1,31 +0,0 @@ -#!/bin/bash -# -# shell.sh -# Opens a bash shell inside the simulator container. -# Creates and starts the simulator if it doesn't already exist. -# - -set -e - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" - -# Check if simulator container exists -if ! docker ps -a --format '{{.Names}}' | grep -q '^simulator$'; then - echo "Simulator container not found, starting it..." - cd "$SCRIPT_DIR" - docker compose up -d - - # Wait for container to be healthy - echo "Waiting for simulator to be ready..." - until docker exec simulator docker info > /dev/null 2>&1; do - sleep 1 - done - echo "Simulator is ready!" -elif ! docker ps --format '{{.Names}}' | grep -q '^simulator$'; then - echo "Simulator exists but is not running, starting it..." - cd "$SCRIPT_DIR" - docker compose up -d -fi - -# Spawn shell in the simulator -exec docker exec -it simulator sh diff --git a/simulator/simulator.yml b/simulator/simulator.yml deleted file mode 100644 index 97cae73..0000000 --- a/simulator/simulator.yml +++ /dev/null @@ -1,58 +0,0 @@ -version: '3.8' - -services: - simulator: - image: docker:27-dind - container_name: simulator - privileged: true - environment: - # Allow insecure registries for local testing - DOCKER_TLS_CERTDIR: "" - volumes: - # Mount the inner docker-compose file - - ./filesystem:/workspace:ro - # Mount the pre-built image tarball (created by prepare-simulator.sh) - - ./filesystem/magic-proxy.tar.gz:/images/magic-proxy.tar.gz:ro - ports: - # Expose the inner Docker daemon (optional, for debugging) - - "2375:2375" - # Expose traefik port from inside the simulator - - "8080:80" - healthcheck: - test: ["CMD", "docker", "info"] - interval: 5s - timeout: 5s - retries: 10 - command: > - sh -c " - dockerd-entrypoint.sh & - - # Wait for Docker daemon to be ready - until docker info > /dev/null 2>&1; do - echo 'Waiting for Docker daemon...' - sleep 1 - done - - echo 'Docker daemon ready' - - # Load the pre-built magic-proxy image if it exists - if [ -f /images/magic-proxy.tar.gz ]; then - echo 'Loading magic-proxy image...' - docker load < /images/magic-proxy.tar.gz - echo 'Image loaded successfully' - else - echo 'Warning: magic-proxy.tar.gz not found, skipping image load' - fi - - # Install docker-compose - apk add --no-cache docker-compose - - # Start the inner environment - cd /workspace - docker-compose -f test-env.yml up -d - - echo 'Simulator environment started' - - # Keep container running and tail logs - docker-compose -f test-env.yml logs -f - " diff --git a/simulator/test.sh b/simulator/test.sh deleted file mode 100755 index fdfabe2..0000000 --- a/simulator/test.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/bash -# -# test.sh -# Test connectivity to the magic-proxy API running inside the simulator. -# Uses docker run with curl to test from within the simulator's network. -# - -set -e - -echo "Testing magic-proxy API running inside the simulator..." -echo "" - -# Try to reach the magic-proxy container -docker exec simulator docker run --rm --network workspace_default alpine:latest wget -q -O- http://workspace-magic-proxy-1:3000/ 2>/dev/null > /dev/null - -if [ $? -eq 0 ]; then - echo "✓ Successfully connected to magic-proxy on port 3000!" - echo "" - echo "Fetching API response:" - docker exec simulator docker run --rm --network workspace_default alpine:latest wget -q -O- http://workspace-magic-proxy-1:3000/ 2>/dev/null || true -else - echo "✗ Failed to connect to magic-proxy" - echo "" - echo "Checking container status..." - docker exec simulator docker ps --filter name=magic-proxy - echo "" - echo "Recent logs:" - docker exec simulator docker logs --tail 20 workspace-magic-proxy-1 - exit 1 -fi diff --git a/simulator/up.sh b/simulator/up.sh deleted file mode 100755 index 4adc9bf..0000000 --- a/simulator/up.sh +++ /dev/null @@ -1,31 +0,0 @@ -#!/bin/bash -# -# up.sh -# Prepare the simulator environment and start it. -# This builds the magic-proxy image, exports it, and starts the docker-compose. -# - -set -e - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" - -echo "=== Starting Test Environment ===" -echo "" - -# First, prepare the image -echo "Step 1: Building and exporting magic-proxy image..." -"$SCRIPT_DIR/prepare-simulator.sh" - -echo "" -echo "Step 2: Starting simulator..." -cd "$SCRIPT_DIR" -docker compose up -d - -echo "" -echo "=== Test Environment Started ===" -echo "" -echo "Available commands:" -echo " ./shell.sh - Open a shell inside the simulator" -echo " ./logs.sh - View logs from magic-proxy inside the simulator" -echo " ./test.sh - Test connectivity to the magic-proxy API" -echo " ./down.sh - Stop and remove the simulator" diff --git a/src/api/server.ts b/src/api/server.ts index 1163abf..5627a3d 100644 --- a/src/api/server.ts +++ b/src/api/server.ts @@ -91,7 +91,7 @@ export async function startAPI(apiConfig: APIConfig): Promise { try { await new Promise((resolve, reject) => { server = app.listen(apiConfig.port, '0.0.0.0', () => { - log.info({ + log.debug({ message: 'Magic Proxy API started', data: { port: apiConfig.port } }); @@ -115,6 +115,6 @@ export function stopAPI(): void { if (server) { server.close(); server = null; - log.info({ message: 'API server stopped' }); + log.debug({ message: 'API server stopped' }); } } diff --git a/src/backends/traefik/templateParser.ts b/src/backends/traefik/templateParser.ts index d16a59d..fbd28f1 100644 --- a/src/backends/traefik/templateParser.ts +++ b/src/backends/traefik/templateParser.ts @@ -4,28 +4,38 @@ import { zone } from '../../logging/zone'; const log = zone('backends.traefik.template'); -/** Pattern for template variables: {{ variable_name }} */ -const VARIABLE_PATTERN = /{{\s*([a-zA-Z0-9_]+)\s*}}/g; +/** Pattern for template variables: {{ variable_name }} or {{ object.property }} */ +const VARIABLE_PATTERN = /{{\s*([a-zA-Z0-9_.]+)\s*}}/g; -/** Pattern for valid variable names */ +/** Pattern for valid userData key names (alphanumeric and underscores only, no dots) */ const VALID_KEY_PATTERN = /^[a-zA-Z0-9_]+$/; /** * Build the context object from app name and proxy data. * Core keys (app_name, hostname, target_url) cannot be overwritten by userData. + * Supports both: + * - Flat keys: {{ port }} (for backward compatibility) + * - Nested access: {{ userData.port }} (explicit namespace) */ -function buildContext(appName: string, data: XMagicProxyData): Record { - const context: Record = { +function buildContext(appName: string, data: XMagicProxyData): Record { + const CORE_KEYS = new Set(['app_name', 'hostname', 'target_url', 'userData']); + + const context: Record = { app_name: appName, hostname: data.hostname, target_url: data.target, + userData: {}, }; - // Merge user-supplied data (skip invalid keys and reserved names) + // Merge user-supplied data into both flat context and userData namespace + // Skip keys that match core variables to prevent overwrites if (data.userData && typeof data.userData === 'object') { for (const [key, value] of Object.entries(data.userData)) { - if (VALID_KEY_PATTERN.test(key) && !(key in context)) { - context[key] = value == null ? '' : String(value); + if (VALID_KEY_PATTERN.test(key) && !CORE_KEYS.has(key)) { + const stringValue = value == null ? '' : String(value); + // Add to both flat keys ({{ port }}) and nested namespace ({{ userData.port }}) + context[key] = stringValue; + context.userData[key] = stringValue; } } } @@ -55,10 +65,29 @@ export function renderTemplate(template: string, appName: string, data: XMagicPr // Track unknown variables const unknownVariables: string[] = []; + /** + * Get a value from context, supporting nested property access with dot notation. + * e.g., "userData.foo" returns context.userData.foo + */ + function getContextValue(path: string): string | undefined { + const parts = path.split('.'); + let value: any = context; + + for (const part of parts) { + if (value == null || typeof value !== 'object') { + return undefined; + } + value = value[part]; + } + + return value == null ? undefined : String(value); + } + // Replace all {{ key }} occurrences const rendered = template.replace(VARIABLE_PATTERN, (_match, key: string) => { - if (key in context) { - return context[key]; + const value = getContextValue(key); + if (value !== undefined) { + return value; } // Track unknown variable for error reporting unknownVariables.push(key); diff --git a/src/backends/traefik/traefik.ts b/src/backends/traefik/traefik.ts index da72420..079a9fc 100644 --- a/src/backends/traefik/traefik.ts +++ b/src/backends/traefik/traefik.ts @@ -41,7 +41,7 @@ async function loadTemplate(templatePath: string): Promise { /** * Creates a Traefik config fragment by rendering the appropriate template. - * Returns null if template rendering fails. + * Returns null if template rendering fails (template not found or render error). */ function makeAppConfig(appName: string, data: XMagicProxyData): TraefikConfigYamlFormat | null { lastUserData = data.userData ? JSON.stringify(data.userData) : null; @@ -50,7 +50,7 @@ function makeAppConfig(appName: string, data: XMagicProxyData): TraefikConfigYam if (!templateContent) { const available = Array.from(templates.keys()).join(', ') || '(none)'; log.error({ message: 'Template not found', data: { appName, template: data.template, available } }); - throw new Error(`Template '${data.template}' not found for app '${appName}'. Available: ${available}`); + return null; } log.debug({ @@ -112,7 +112,7 @@ export async function initialize(config?: MagicProxyConfigFile): Promise { throw new Error('No templates defined in config.traefik.templates'); } - log.info({ message: 'Initializing Traefik backend', data: { templateCount: templatePaths.length } }); + log.debug({ message: 'Initializing Traefik backend', data: { templateCount: templatePaths.length } }); // Load all templates concurrently const loadResults = await Promise.all( @@ -139,7 +139,7 @@ export async function initialize(config?: MagicProxyConfigFile): Promise { ? outputFile : path.resolve(OUTPUT_DIRECTORY, outputFile); manager.setOutputFile(resolved); - log.info({ message: 'Output file configured', data: { outputFile: resolved } }); + log.debug({ message: 'Output file configured', data: { outputFile: resolved } }); } await manager.flushToDisk(); diff --git a/src/backends/traefik/traefikManager.ts b/src/backends/traefik/traefikManager.ts index 399290e..c62140f 100644 --- a/src/backends/traefik/traefikManager.ts +++ b/src/backends/traefik/traefikManager.ts @@ -196,12 +196,6 @@ async function doFlushToDisk(): Promise { throw new Error(`Invalid config generated: ${validation.error}`); } - if (validation.warnings?.length) { - for (const warning of validation.warnings) { - log.warn({ message: warning }); - } - } - await writeAtomically(outputFile, yamlText); } diff --git a/src/backends/traefik/validators.ts b/src/backends/traefik/validators.ts index 1ccd6de..45d3238 100644 --- a/src/backends/traefik/validators.ts +++ b/src/backends/traefik/validators.ts @@ -16,7 +16,7 @@ const ALLOWED_UDP_KEYS = new Set(['services']); const INVALID_NAME_PATTERN = /\s|\n/; export type ValidationResult = - | { valid: true; warnings?: string[] } + | { valid: true } | { valid: false; error: string }; /** @@ -51,11 +51,10 @@ function validateSectionKeys( /** * Validate generated Traefik configuration YAML. - * Returns validation result with optional warnings. + * Returns validation result. Since renderTemplate now throws on unknown variables, + * no unreplaced template variables should ever reach this validator. */ export function validateGeneratedConfig(yamlText: string): ValidationResult { - const warnings: string[] = []; - // Parse YAML let parsed: unknown; try { @@ -116,10 +115,5 @@ export function validateGeneratedConfig(yamlText: string): ValidationResult { if (nameError) return { valid: false, error: nameError }; } - // Check for unreplaced template variables - if (yamlText.includes('{{') || yamlText.includes('}}')) { - warnings.push('Config contains unreplaced template variables (may indicate missing data)'); - } - - return { valid: true, warnings: warnings.length > 0 ? warnings : undefined }; + return { valid: true }; } \ No newline at end of file diff --git a/src/index.ts b/src/index.ts index d2fc01f..2f6349f 100644 --- a/src/index.ts +++ b/src/index.ts @@ -8,7 +8,7 @@ import { startWatchingConfigFile, resetRestartFlag } from './configWatcher'; const log = zone('index'); -log.info({ +log.debug({ message: 'Starting Magic Proxy application', }); @@ -34,7 +34,7 @@ export async function startApp(config?: MagicProxyConfigFile) { dockerProvider = new DockerProvider(hostDb); await dockerProvider.start(); - log.info({ + log.debug({ message: 'Docker provider started - monitoring for container changes' }); @@ -71,7 +71,7 @@ export async function startApp(config?: MagicProxyConfigFile) { * Handler called when config file changes */ async function handleConfigChange(newConfig: MagicProxyConfigFile): Promise { - log.info({ + log.debug({ message: 'Config file changed - restarting application' }); diff --git a/src/providers/docker/manifest.ts b/src/providers/docker/manifest.ts index 771844b..b763368 100644 --- a/src/providers/docker/manifest.ts +++ b/src/providers/docker/manifest.ts @@ -78,7 +78,7 @@ export function logManifestSummary(results: ProcessingResult): void { (sum, r) => sum + Object.values(r).filter(s => s === 'ok').length, 0 ); - log.info({ + log.debug({ message: 'Processed container(s)', data: { total, added, skippedOrFailed: total - added } }); diff --git a/src/providers/docker/provider.ts b/src/providers/docker/provider.ts index 0518bde..1496e05 100644 --- a/src/providers/docker/provider.ts +++ b/src/providers/docker/provider.ts @@ -44,13 +44,13 @@ export class DockerProvider { } this.isActive = true; - log.info({ message: 'Starting Docker provider' }); + log.debug({ message: 'Starting Docker provider' }); await this.syncDatabase(); this.watchDockerEvents(); await this.updateFileWatchers(); - log.info({ message: 'Docker provider started successfully' }); + log.debug({ message: 'Docker provider started successfully' }); } /** @@ -59,7 +59,7 @@ export class DockerProvider { stop(): void { if (!this.isActive) return; - log.info({ message: 'Stopping Docker provider' }); + log.debug({ message: 'Stopping Docker provider' }); this.isActive = false; // Clean up Docker event stream @@ -76,7 +76,7 @@ export class DockerProvider { } this.fileWatchers.clear(); - log.info({ message: 'Docker provider stopped' }); + log.debug({ message: 'Docker provider stopped' }); } /** @@ -134,7 +134,7 @@ export class DockerProvider { const syncActions = ['create', 'start', 'destroy', 'die', 'stop']; if (syncActions.includes(action)) { - log.info({ message: `Container ${action}`, data: { containerName, id } }); + log.debug({ message: `Container ${action}`, data: { containerName, id } }); this.scheduleSync(); } } @@ -147,7 +147,7 @@ export class DockerProvider { setTimeout(() => { if (this.isActive) { - log.info({ message: 'Reconnecting to Docker event stream' }); + log.debug({ message: 'Reconnecting to Docker event stream' }); this.watchDockerEvents(); } }, 5000); @@ -173,7 +173,7 @@ export class DockerProvider { data: { path, eventType, filename, isActive: this.isActive } }); - log.info({ message: 'Compose file changed', data: { path, eventType } }); + log.debug({ message: 'Compose file changed', data: { path, eventType } }); // On rename events (atomic writes), re-attach the watcher // because the original inode may have been replaced @@ -287,13 +287,13 @@ export class DockerProvider { * Synchronize the database with current Docker state */ private async syncDatabase(): Promise { - log.info({ message: 'Starting database sync' }); + log.debug({ message: 'Starting database sync' }); try { const { manifest } = await buildContainerManifest(this.docker); const manifestNames = new Set(manifest.map(e => e.containerName)); - log.info({ + log.debug({ message: 'Manifest built', data: { containerCount: manifest.length, @@ -358,7 +358,7 @@ export class DockerProvider { let entriesRemoved = 0; for (const entry of this.hostDb.getAll()) { if (!manifestNames.has(entry.containerName)) { - log.info({ + log.debug({ message: 'Removing container no longer referenced', data: { containerName: entry.containerName } }); @@ -370,7 +370,7 @@ export class DockerProvider { // Log if file change resulted in no database updates const totalChanges = entriesAdded + entriesUpdated + entriesRemoved; if (totalChanges === 0 && manifest.length > 0) { - log.info({ + log.debug({ message: 'Database sync completed with no changes', data: { manifestCount: manifest.length, diff --git a/test-file-changes.sh b/test-file-changes.sh deleted file mode 100755 index f755348..0000000 --- a/test-file-changes.sh +++ /dev/null @@ -1,69 +0,0 @@ -#!/bin/bash -# Test script to verify repeated compose file changes trigger updates - -set -e - -echo "=== Testing Repeated Compose File Changes ===" -echo "" - -# Colors for output -GREEN='\033[0;32m' -BLUE='\033[0;34m' -NC='\033[0m' # No Color - -TEST_FILE="/tmp/test-compose-changes.yml" - -# Create a test compose file -cat > "$TEST_FILE" << 'EOF' -version: "3.9" -services: - test-app: - image: nginx - x-magic-proxy: - template: oidc.yml - hostname: test.example.com - target: http://10.0.0.1:8080 -EOF - -echo -e "${BLUE}Initial file content:${NC}" -grep "target:" "$TEST_FILE" -echo "" - -# Change 1 -echo -e "${GREEN}Change 1: Updating target to 10.0.0.2${NC}" -sed -i 's/10.0.0.1/10.0.0.2/g' "$TEST_FILE" -grep "target:" "$TEST_FILE" -sleep 1 -echo "" - -# Change 2 -echo -e "${GREEN}Change 2: Updating target to 10.0.0.3${NC}" -sed -i 's/10.0.0.2/10.0.0.3/g' "$TEST_FILE" -grep "target:" "$TEST_FILE" -sleep 1 -echo "" - -# Change 3 -echo -e "${GREEN}Change 3: Updating target to 10.0.0.4${NC}" -sed -i 's/10.0.0.3/10.0.0.4/g' "$TEST_FILE" -grep "target:" "$TEST_FILE" -sleep 1 -echo "" - -# Change back -echo -e "${GREEN}Change 4: Changing back to 10.0.0.1${NC}" -sed -i 's/10.0.0.4/10.0.0.1/g' "$TEST_FILE" -grep "target:" "$TEST_FILE" -echo "" - -echo -e "${BLUE}Test complete! Each change should have been detected.${NC}" -echo "Check the application logs to verify all 4 changes triggered updates." -echo "" -echo "Expected behavior:" -echo " - Changes 1, 2, 3, 4 should all trigger 'Compose file changed' logs" -echo " - Changes 1, 2, 3, 4 should all trigger backend updates (target changed)" -echo " - The hostDb should show the updated target each time" -echo "" -echo "If you see updates for change 1 but not 2-4, there may be an issue." - -rm "$TEST_FILE" diff --git a/test/unit/traefik/template-validation.test.ts b/test/unit/traefik/template-validation.test.ts index ed4208b..c5f3995 100644 --- a/test/unit/traefik/template-validation.test.ts +++ b/test/unit/traefik/template-validation.test.ts @@ -202,8 +202,12 @@ http: expect(result.valid === false && result.error).toContain('Invalid name'); }); - it('warns about unreplaced template variables', () => { - // Use plain text that contains template markers + it('no longer warns about unreplaced template variables - they error at render time', () => { + // Unreplaced template variables are now caught by renderTemplate() before + // they reach the validator. This test verifies the validator doesn't see them. + // If someone manually creates YAML with template markers (which shouldn't happen + // through normal flow), they would still be allowed by the validator since + // template syntax is valid in string values. const yaml = ` http: routers: @@ -213,27 +217,28 @@ http: services: my-service: {} middlewares: - test: "contains {{ app_name }} variable" + test: "contains {{ app_name }} variable in a string" `; const result = validateGeneratedConfig(yaml); + // This should be valid - templates in string values are fine expect(result.valid).toBe(true); - expect(result.valid === true && result.warnings?.length).toBeGreaterThan(0); - expect(result.valid === true && result.warnings?.[0]).toContain('unreplaced template'); }); - it('warns about unreplaced variables with }}', () => { + it('accepts template syntax in string values without warnings', () => { + // Template markers are valid in string values - they're just text const yaml = ` http: routers: app: - rule: Host(\`{{ hostname }}\`) + rule: Host(\`example.com\`) service: my-service services: my-service: {} + middlewares: + documentation: "Use {{ variable }} syntax in templates" `; const result = validateGeneratedConfig(yaml); expect(result.valid).toBe(true); - expect(result.valid === true && result.warnings?.length).toBeGreaterThan(0); }); }); @@ -290,13 +295,6 @@ http: const validation = validateGeneratedConfig(rendered); expect(validation.valid).toBe(true); - // Should not have warnings about unreplaced variables - if (validation.valid && validation.warnings) { - const unreplacedWarning = validation.warnings.find(w => - w.includes('unreplaced template') - ); - expect(unreplacedWarning).toBeUndefined(); - } }); it('validates complex template with multiple apps', () => { diff --git a/test/unit/traefik/user-data-substitution.test.ts b/test/unit/traefik/user-data-substitution.test.ts index 5391149..68c424f 100644 --- a/test/unit/traefik/user-data-substitution.test.ts +++ b/test/unit/traefik/user-data-substitution.test.ts @@ -214,6 +214,48 @@ config: expect(result).toContain('val2'); expect(result).toContain('3000'); }); + + it('rejects userData keys with dots to prevent ambiguity', () => { + const template = ` +config: + value: {{ foo }} +`; + const data: XMagicProxyData = { + template: 'test', + target: 'http://backend:3000', + hostname: 'example.com', + userData: { + 'foo.bar': 'should-be-ignored', + 'foo': 'correct-value', + }, + }; + + // The userData key with a dot in it ('foo.bar') should not be added to context + // because dots are reserved for template variable syntax (nested access) + const result = renderTemplate(template, 'app', data); + expect(result).toContain('correct-value'); + expect(result).not.toContain('should-be-ignored'); + }); + + it('supports nested userData access with dots in template syntax', () => { + const template = ` +config: + value: {{ userData.my_value }} +`; + const data: XMagicProxyData = { + template: 'test', + target: 'http://backend:3000', + hostname: 'example.com', + userData: { + my_value: 'nested-access-works', + }, + }; + + // Even though userData keys can't have dots, template syntax supports nested access + // so {{ userData.my_value }} should work + const result = renderTemplate(template, 'app', data); + expect(result).toContain('nested-access-works'); + }); }); describe('Complex template scenarios with userData', () => {