Skip to content

feat: Atomic backup mode for single snapshot for volumes + databases#111

Open
muebau wants to merge 1 commit intolawndoc:mainfrom
muebau:atomic-backups
Open

feat: Atomic backup mode for single snapshot for volumes + databases#111
muebau wants to merge 1 commit intolawndoc:mainfrom
muebau:atomic-backups

Conversation

@muebau
Copy link

@muebau muebau commented Mar 17, 2026

solves #110

Problem

Currently, volumes and database dumps are backed up as separate restic snapshots. This means a restore requires picking the right volume snapshot and the matching database snapshot(s), with no guarantee they represent the same point in time. For applications like Nextcloud, where data directory and database must be consistent, this is a real risk.

Solution

A new environment variable ATOMIC_BACKUP (true/false, default false).

When enabled, the backup process:

  1. Stops containers labeled stop-during-backup (existing behavior)
  2. Dumps each database to a file inside /databases/ on the backup container's filesystem (instead of piping directly into restic backup --stdin)
  3. Runs a single restic backup /volumes /databases call — one snapshot containing everything
  4. Cleans up the /databases/ dump files
  5. Restarts stopped containers

When disabled (default), the existing behavior is fully preserved — no breaking changes.

Usage

backup:
  image: ghcr.io/lawndoc/stack-back
  environment:
    ATOMIC_BACKUP: "true"
    # ... other env vars

The resulting restic snapshot contains:

/volumes/{service}/{path}/...   ← volume data
/databases/{service}/all_databases.sql   ← SQL dumps

A single restic restore retrieves everything needed to reconstruct the stack.

Changes

File What changed
config.py Read ATOMIC_BACKUP env var
commands.py New docker_exec_to_file() — streams docker exec stdout to a local file
containers_db.py New dump_to_file() on MariadbContainer, MysqlContainer, PostgresContainer
containers.py dump_to_file() abstract method on base Container
restic.py backup_files() now accepts a list of source paths (backward-compatible)
cli.py start_backup_process() branches into atomic/standard mode; status() reports the setting
fixtures.py Include Env in container fixture (needed for DB credential tests)
test_atomic_backup.py 12 new unit tests covering config, dump paths, delegation, multi-source backup

Testing

All existing unit tests continue to pass. 12 new tests cover:

  • ATOMIC_BACKUP config: default off, enabled via "true", enabled via "1"
  • Dump destination paths: all three DB types write under /databases/
  • dump_to_file() delegation: each DB subclass calls docker_exec_to_file with correct container ID, dump command, file path, and credentials
  • backup_files() multi-source: single string, list of paths, default value

Backward compatibility

  • Default behavior (ATOMIC_BACKUP unset or false) is unchanged — volumes and databases remain separate snapshots
  • restic.backup_files() still accepts a single string — no caller changes needed
  • No new dependencies

…ackups

When ATOMIC_BACKUP=true, database dumps are written to /databases/ on the
backup container's filesystem and then backed up together with /volumes in
a single `restic backup` call, producing one atomic snapshot. This ensures
volumes and database dumps are always consistent and restorable as a unit.

Without the flag (default), the existing behavior is preserved: volumes
and each database are backed up as separate restic snapshots.

Changes:
- config.py: read ATOMIC_BACKUP env var
- commands.py: add docker_exec_to_file() to stream docker exec stdout
  to a local file
- containers_db.py: add dump_to_file() to MariaDB, MySQL, PostgreSQL
  container subclasses
- containers.py: add dump_to_file() abstract method to base Container
- restic.py: backup_files() now accepts a list of source paths
- cli.py: start_backup_process() branches into atomic/standard mode;
  status() reports atomic backup setting
- fixtures.py: include Env in container fixture for DB credential tests
- test_atomic_backup.py: 12 unit tests covering config, dump paths,
  dump_to_file delegation, and multi-source backup_files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant