diff --git a/README.md b/README.md index 28a614e..0493853 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,8 @@ A Bash script to generate a tar.gz backup of a folder, with an option to automatically upload the backup file to a cloud service using [rclone](https://github.com/rclone/rclone). List of cloud/storage providers currently supported by rclone can be found [here](https://github.com/rclone/rclone#storage-providers). -
-Latest version: 1.2.0 ([changelog](https://github.com/MichaelYochpaz/Backup-Script/blob/master/changelog.md)) + +Latest version: 1.3.0 ([changelog](changelog.md)) ## Features * Generate a tar.gz backup file of a folder. @@ -23,10 +23,16 @@ Latest version: 1.2.0 ([changelog](https://github.com/MichaelYochpaz/Backup-Scri Configuration can be set on the script file under the "Configuration" section, or using command-line arguments (will override configuration that's set on on the script). ``` - Usage: backup [-n ] [-s ] [-e ]... [-u ] [-r] [-v] + Usage: backup [-n ] [-d ] [-l ] [-s ] [-e ]... [-u ] [-p ] [-r] [-y] [-v] Options: -n Sets the tar.gz file name [default: "backup"] + -d Sets date command format [default: "%FT%H%M" ie: "2024-03-05T0701"] + -l Incremental backup mode: Adds 'level' to the filename. + If the number is greater than 0, an incremental backup is performed. + Full backups are performed by default (and when a snar file does not exist), + but indicating a level will create a snar file that will store data + necessary to perform future backups. -s Path to which the generated backup file will be saved to [default: current working directory] -e Exclude a pattern (specific files / folders) from being backed up -u rclone path to which the backup file will be uploaded to (not providing one will skip the upload process) @@ -42,6 +48,7 @@ or using command-line arguments (will override configuration that's set on on th backup "/home/user/important_stuff" backup -u "GDrive:/Backups" -r -y -p "XXXXXXXXXXXXXXXX" "/home/user/important_stuff/" backup -n "important-stuff-backup" -s "/home/user/backups" -e "*.pdf" -e "important_stuff/dont_backup_this_folder" "/home/user/important_stuff/" + backup -l 0 -s /mnt/backup /home/michael ``` ## FAQ @@ -52,4 +59,4 @@ or using command-line arguments (will override configuration that's set on on th **A:** Go to your Pushbullet account's [settings page](https://www.pushbullet.com/#settings/account), and click the "Create Access Token" button. **Q:** I found a bug, or have an idea for a feature. How can I help? -**A:** Feel free to open an [issue](https://github.com/MichaelYochpaz/Backup-Script/issues) and post your issue / suggestion. \ No newline at end of file +**A:** Feel free to open an issue and post your issue / suggestion. diff --git a/backup b/backup old mode 100644 new mode 100755 index 4371e5a..60fa650 --- a/backup +++ b/backup @@ -1,8 +1,9 @@ #!/bin/bash ## ------------------------------------ ## Made By: Michael Yochpaz (C) 2020 -## https://github.com/MichaelYochpaz/Backup-Script -## Version: 1.2.0 +## Contributor: Michael Soh (github:sohmc) +## https://github.com/sohmc/Backup-Script +## Version: 1.3.0 ## License: GPLv3 ## ------------------------------------ ## -------------- Usage --------------- @@ -13,10 +14,16 @@ ## - In order to use the upload feature, rclone must be installed and have at least one cloud service setup in config. ## ------------------------------------ ## -## Usage: backup [-n ] [-s ] [-e ]... [-u ] [-p ] [-r] [-y] [-v] +## Usage: backup [-n ] [-d ] [-l ] [-s ] [-e ]... [-u ] [-p ] [-r] [-y] [-v] ## ## Options: ## -n Sets the tar.gz file name [default: "backup"] +## -d Sets date command format [default: "%FT%H%M" ie: "2024-03-05T0701"] +## -l Incremental backup mode: Adds 'level' to the filename. +## If the number is greater than 0, an incremental backup is performed. +## Full backups are performed by default (and when a snar file does not exist), +## but indicating a level will create a snar file that will store data +## necessary to perform future backups. ## -s Path to which the generated backup file will be saved to [default: current working directory] ## -e Exclude a pattern (specific files / folders) from being backed up ## -u rclone path to which the backup file will be uploaded to (not providing one will skip the upload process) @@ -44,13 +51,15 @@ REMOVE_LOCAL=false # Boolean - Remove local backup file after upload. SKIP_WARNINGS=false # Boolean - Skip warnings. VERBOSE=false # Boolean - Use verbose mode when running tar and rclone. PUSHBULLET_TITLE="Backup Script" # Pusbullet - Notification title. +DATE_FORMAT='%Y%m%d-%H%M%S' +INCREMENTAL_LEVEL=-1 PUSHBULLET_MESSAGE_START="Backup script " # Pushbullet - Notification messages will start with this string, followed by one of the followings: PUSHBULLET_MESSAGE_FINISHED_SUCCESS="finished successfully." # Pusbullet - Message will be shown if the script ran successfully. PUSHBULLET_MESSAGE_FINISHED_ERRORS="finished with errors." # Pusbullet - Message will be shown if the script finished with errors. PUSHBULLET_MESSAGE_FAIL="failed." # Pusbullet - Message will be shown if the script has failed. ## ------------------------------------ -usage() { echo "Usage: $(basename $0) [-n ] [-s ] [-e ]... [-u ] [-p ] [-r] [-y] [-v] +usage() { echo "Usage: $(basename $0) [-n ] [-d ] [-l ] [-s ] [-e ]... [-u ] [-p ] [-r] [-y] [-v] Use $(basename $0) -h for additional info."; exit 1; } # if there is a Pushbullet API key, send a Pushbullet notification. uses a parameter for message's body @@ -65,11 +74,17 @@ C2='\033[1;31m' # ANSI - error - red C3='\033[0;36m' # ANSI - information - cyan C4='\033[0;33m' # ANSI - variables/paths - orange -while getopts ":n:e:s:u:p:ryvh" arg; do +while getopts ":n:d:l:e:s:u:p:ryvh" arg; do case $arg in n) BACKUP_NAME=${OPTARG} ;; + d) + DATE_FORMAT=${OPTARG} + ;; + l) + INCREMENTAL_LEVEL=${OPTARG} + ;; e) EXCLUDES+=("${OPTARG}") ;; @@ -92,9 +107,15 @@ while getopts ":n:e:s:u:p:ryvh" arg; do VERBOSE=true ;; h) - echo " Usage: $(basename $0) [-n ] [-s ] [-e ]... [-u ] [-p ] [-r] [-y] [-v] + echo " Usage: $(basename $0) [-n ] [-d ] [-l ] [-s ] [-e ]... [-u ] [-p ] [-r] [-y] [-v] Options: -n Sets the tar.gz file name [default: "backup"] + -d Sets date command format [default: "%FT%H%M" ie: "2024-03-05T0701"] + -l Incremental backup mode: Adds 'level' to the filename. + If the number is greater than 0, an incremental backup is performed. + Full backups are performed by default (and when a snar file does not exist), + but indicating a level will create a snar file that will store data + necessary to perform future backups. -s Path to which the generated backup file will be saved to [default: current working directory] -e Exclude a pattern (specific files / folders) from being backed up -u rclone path to which the backup file will be uploaded to (not providing one will skip the upload process) @@ -149,19 +170,70 @@ if [ "$REMOVE_LOCAL" = true ] && [ -z "$RCLONE_FOLDER" ] && [] "$SKIP_WARNINGS" if [[ ! $REPLY =~ ^[Yy]$ ]]; then exit 0; fi fi +# Create Kebabed folder name so that we can use it as a file name +KEBAB_BACKUP_FOLDER=${BACKUP_FOLDER//\//-} +# Remove dashes or dots if they are in the front of the kebab +shopt -s extglob +KEBAB_BACKUP_FOLDER="${KEBAB_BACKUP_FOLDER##+([.-])}" +shopt -u extglob +# if INCREMENTAL_LEVEL is GTE 0, enable incremental label +INCREMENTAL_LABEL="" +INCREMENTAL_FLAG="" +if [ ${INCREMENTAL_LEVEL} -ge 0 ]; then + TAR_CACHE=$HOME/.cache/tar + SNAR_FILE=$TAR_CACHE/$KEBAB_BACKUP_FOLDER.snar + INCREMENTAL_LABEL="-level${INCREMENTAL_LEVEL}" + INCREMENTAL_FLAG="--listed-incremental=${SNAR_FILE}" + + echo -e "$(date +"%Y/%m/%d %T") ${C3}Incremental level set to $INCREMENTAL_LEVEL"${C0} + + if [ ! -d $TAR_CACHE ]; then + echo -e "$(date +"%Y/%m/%d %T") ${C3}Creating ${C4}$TAR_CACHE ${C3} directory"${C0} + mkdir -p $TAR_CACHE + fi + + if [ -f $SNAR_FILE ]; + then echo -e "$(date +"%Y/%m/%d %T") ${C3}snar file ${C4}$SNAR_FILE ${C3}found"${C0} + else + echo -e "$(date +"%Y/%m/%d %T") ${C3}snar file ${C4}$SNAR_FILE ${C3}not found. It will be created and a full backup will be performed."${C0} + fi + + if [ ${INCREMENTAL_LEVEL} -eq 0 ] && [ -f $SNAR_FILE ]; then + echo -e "$(date +"%Y/%m/%d %T") ${C3}Incremental Level set to 0 and snar file exists. Deleting..."${C0} + rm $SNAR_FILE + fi +fi + # ----- tar ----- echo -e "$(date +"%Y/%m/%d %T") ${C3}Generating tar.gz file"${C0} -FILENAME="${BACKUP_NAME}-$(date +%FT%H%M).tar.gz" +FILENAME="${BACKUP_NAME}-$(date +${DATE_FORMAT})${INCREMENTAL_LABEL}.tar.gz" +echo -e "$(date +"%Y/%m/%d %T") ${C3}tar FILENAME: ${C4}${FILENAME}"${C0} # generate a temporary file to store EXCLUDES in -EXCLUEDS_FILE=$(mktemp /tmp/backup.XXXXXX.txt) -if [ ! ${#EXCLUDES[@]} -eq 0 ]; then printf "%s\n" "${EXCLUDES[@]}" > "$EXCLUEDS_FILE"; fi +EXCLUDES_FILE=$(mktemp /tmp/backup.XXXXXX.txt) +if [ ! ${#EXCLUDES[@]} -eq 0 ]; then printf "%s\n" "${EXCLUDES[@]}" > "$EXCLUDES_FILE"; fi + +# Include default.exclude if it exists +DEFAULT_EXCLUDE=$HOME/.config/tar/default.exclude +if [ -f $DEFAULT_EXCLUDE ]; then + echo -e "$(date +"%Y/%m/%d %T") ${C3} Default exclude file found. Adding to excludes..."${C0} + cat $DEFAULT_EXCLUDE >> "$EXCLUDES_FILE"; +fi +# Include a {kebabbed}.exclude if it exists +KEBAB_BACKUP_FOLDER_EXCLUDE=$HOME/.config/tar/$KEBAB_BACKUP_FOLDER.exclude +if [ -f $KEBAB_BACKUP_FOLDER_EXCLUDE ]; then + echo -e "$(date +"%Y/%m/%d %T") ${C3} Exclude file for ${KEBAB_BACKUP_FOLDER} found. Adding to excludes..."${C0} + cat $KEBAB_BACKUP_FOLDER_EXCLUDE >> "$EXCLUDES_FILE"; +fi + # run tar with or without '-v', according to VERBOSE value -if [ "$VERBOSE" = true ]; then tar -cvzf "$SAVE_FOLDER/$FILENAME" -C "$BACKUP_FOLDER/.." -X $EXCLUEDS_FILE "$(basename "$BACKUP_FOLDER")"; \ -else tar -czf "$SAVE_FOLDER/$FILENAME" -C "$BACKUP_FOLDER/.." -X $EXCLUEDS_FILE "$(basename "$BACKUP_FOLDER")"; fi +VERBOSE_FLAG=""; +if [ "$VERBOSE" = true ]; then VERBOSE_FLAG="-v"; fi + +tar -cz $VERBOSE_FLAG -f "$SAVE_FOLDER/$FILENAME" $INCREMENTAL_FLAG -C "$BACKUP_FOLDER/.." -X $EXCLUDES_FILE "$(basename "$BACKUP_FOLDER")"; # remove temporary file -rm $EXCLUEDS_FILE +rm $EXCLUDES_FILE # tar commmand finished successfully (exit code = 0) if [ "$?" = 0 ]; then @@ -179,7 +251,7 @@ fi if [ -n "$RCLONE_FOLDER" ]; then echo -e "$(date +"%Y/%m/%d %T") ${C3}Uploading backup file to ${C4}$RCLONE_FOLDER${C0}" # run rclone with or without '-v', according to VERBOSE value - if [ "$VERBOSE" = true ]; then rclone copy "$SAVE_FOLDER/$FILENAME" -v "$RCLONE_FOLDER"; else rclone copy "$SAVE_FOLDER/$FILENAME" "$RCLONE_FOLDER"; fi + rclone copy "$SAVE_FOLDER/$FILENAME" $VERBOSE_FLAG "$RCLONE_FOLDER" # rclone commmand finished successfully (exit code = 0) if [ "$?" = 0 ]; then echo -e "$(date +"%Y/%m/%d %T") ${C1}Backup file uploaded successfully${C0}" @@ -214,4 +286,4 @@ else echo -e "$(date +"%Y/%m/%d %T") ${C2}Script finished with errors${C0}" pushbullet "$PUSHBULLET_MESSAGE_FINISHED_ERRORS" exit 1 -fi \ No newline at end of file +fi diff --git a/changelog.md b/changelog.md index 6c0174e..be48a16 100644 --- a/changelog.md +++ b/changelog.md @@ -1,6 +1,12 @@ # Changelog All notable changes to the script will be documented here +## 1.3.0 - [2024-12-23] +* Added feature to set your own `date` format via `-d` argument, followed by a valid format +* Added feature to do incremential backups via the `-l` argument followed by an integer zero or greater +* Script will look for a default exclude list, which is provided within this repo that's been shamelessly copied from [Timeshift](https://github.com/linuxmint/timeshift/blob/07f5589ecd1ff07036fa4a813292a2b5f51baefd/src/Core/Main.vala#L538) +* Relatedly, the script will look for an exclude list for your dataset. Exclude files are stored in `$HOME/.config/tar/$DATASET_NAME.exclude`. Dataset name is derived by replacing slashes in the dataset name with a dash, removing any dots or dashes in the beginning of the dataset name. Best way to use this is to provide the absolute path for the dataset. + ## 1.2.0 - [2020-06-18] * Added Pushbullet notifications support using the `-p` argument * Added `-y` argument to skip warnings (which by default, require user input to continue) diff --git a/default.exclude b/default.exclude new file mode 100644 index 0000000..206c227 --- /dev/null +++ b/default.exclude @@ -0,0 +1,59 @@ +dev/* +proc/* +sys/* +media/* +mnt/* +tmp/* +run/* +var/run/* +var/lock/* +var/lib/dhcpcd/* +var/lib/docker/* +var/lib/schroot/* +lost+found +timeshift/* +timeshift-btrfs/* +data/* +DATA/* +cdrom/* +sdcard/* +system/* +etc/timeshift.json +var/log/timeshift/* +var/log/timeshift-btrfs/* +swapfile +snap/* +root/.thumbnails +root/.cache +root/.dbus +root/.gvfs +root/.local/share/[Tt]rash +home/*/.thumbnails +home/*/.cache +home/*/.dbus +home/*/.gvfs +home/*/.local/share/[Tt]rash +home/*/.thumbnails +home/*/.cache +home/*/.dbus +home/*/.gvfs +home/*/.local/share/[Tt]rash +root/.mozilla/firefox/*.default/Cache +root/.mozilla/firefox/*.default/OfflineCache +root/.opera/cache +root/.kde/share/apps/kio_http/cache +root/.kde/share/cache/http +home/*/.mozilla/firefox/*.default/Cache +home/*/.mozilla/firefox/*.default/OfflineCache +home/*/.vscode-server +home/*/.opera/cache +home/*/.kde/share/apps/kio_http/cache +home/*/.kde/share/cache/http +var/cache/apt/archives/* +var/cache/pacman/pkg/* +var/cache/yum/* +var/cache/dnf/* +var/cache/eopkg/* +var/cache/xbps/* +var/cache/zypp/* +var/cache/edb/* \ No newline at end of file