feat: Atomic backup mode for single snapshot for volumes + databases#111
Open
muebau wants to merge 1 commit intolawndoc:mainfrom
Open
feat: Atomic backup mode for single snapshot for volumes + databases#111muebau wants to merge 1 commit intolawndoc:mainfrom
muebau wants to merge 1 commit intolawndoc:mainfrom
Conversation
…ackups When ATOMIC_BACKUP=true, database dumps are written to /databases/ on the backup container's filesystem and then backed up together with /volumes in a single `restic backup` call, producing one atomic snapshot. This ensures volumes and database dumps are always consistent and restorable as a unit. Without the flag (default), the existing behavior is preserved: volumes and each database are backed up as separate restic snapshots. Changes: - config.py: read ATOMIC_BACKUP env var - commands.py: add docker_exec_to_file() to stream docker exec stdout to a local file - containers_db.py: add dump_to_file() to MariaDB, MySQL, PostgreSQL container subclasses - containers.py: add dump_to_file() abstract method to base Container - restic.py: backup_files() now accepts a list of source paths - cli.py: start_backup_process() branches into atomic/standard mode; status() reports atomic backup setting - fixtures.py: include Env in container fixture for DB credential tests - test_atomic_backup.py: 12 unit tests covering config, dump paths, dump_to_file delegation, and multi-source backup_files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
solves #110
Problem
Currently, volumes and database dumps are backed up as separate restic snapshots. This means a restore requires picking the right volume snapshot and the matching database snapshot(s), with no guarantee they represent the same point in time. For applications like Nextcloud, where data directory and database must be consistent, this is a real risk.
Solution
A new environment variable
ATOMIC_BACKUP(true/false, defaultfalse).When enabled, the backup process:
stop-during-backup(existing behavior)/databases/on the backup container's filesystem (instead of piping directly intorestic backup --stdin)restic backup /volumes /databasescall — one snapshot containing everything/databases/dump filesWhen disabled (default), the existing behavior is fully preserved — no breaking changes.
Usage
The resulting restic snapshot contains:
A single
restic restoreretrieves everything needed to reconstruct the stack.Changes
config.pyATOMIC_BACKUPenv varcommands.pydocker_exec_to_file()— streams docker exec stdout to a local filecontainers_db.pydump_to_file()onMariadbContainer,MysqlContainer,PostgresContainercontainers.pydump_to_file()abstract method on baseContainerrestic.pybackup_files()now accepts a list of source paths (backward-compatible)cli.pystart_backup_process()branches into atomic/standard mode;status()reports the settingfixtures.pyEnvin container fixture (needed for DB credential tests)test_atomic_backup.pyTesting
All existing unit tests continue to pass. 12 new tests cover:
ATOMIC_BACKUPconfig: default off, enabled via"true", enabled via"1"/databases/dump_to_file()delegation: each DB subclass callsdocker_exec_to_filewith correct container ID, dump command, file path, and credentialsbackup_files()multi-source: single string, list of paths, default valueBackward compatibility
ATOMIC_BACKUPunset orfalse) is unchanged — volumes and databases remain separate snapshotsrestic.backup_files()still accepts a single string — no caller changes needed