From 69e21d586caee0e31b40528fff939f2b168e02b0 Mon Sep 17 00:00:00 2001 From: Gerard Bosch <30733556+gerardbosch@users.noreply.github.com> Date: Tue, 12 Dec 2023 15:35:13 +0100 Subject: [PATCH] feat: Log the backup stats summary to a file (#128) * docs: Fix typos and minor improvements * feat: Log the backup stats summary to a CSV file The log records the added, removed and snapshot size after each backup. * feat: Add the snapshot ID to the stats log * docs: Update README with the stats log info * fix: Linter 1: Quote var * fix: Linter 2: Shellcheck declare and assign separately * feat: Turn the stats log into an opt-in --- README.md | 21 +++++++++----- bin/restic_backup.sh | 61 +++++++++++++++++++++++++++------------ etc/restic/_global.env.sh | 8 +++-- 3 files changed, 60 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 6f74b41..0c53aa2 100644 --- a/README.md +++ b/README.md @@ -26,16 +26,16 @@ # Intro [restic](https://restic.net/) is a command-line tool for making backups, the right way. Check the official website for a feature explanation. As a storage backend, I recommend [Backblaze B2](https://www.backblaze.com/b2/cloud-storage.html) as restic works well with it, and it is (at the time of writing) very affordable for the hobbyist hacker! (anecdotal: I pay for my full-systems backups each month typically < 1 USD). -Unfortunately restic does not come pre-configured with a way to run automated backups, say every day. However it's possible to set this up yourself using built-in tools in your OS and some wrappers. For Linux with systemd, it's convenient to use systemd timers. For macOS systems, we can use built-in LaunchAgents. For Windows we can use ScheduledTasks. Any OS having something cron-like will also work! +Unfortunately restic does not come pre-configured with a way to run automated backups, say every day. However, it's possible to set this up yourself using built-in tools in your OS and some wrappers. For Linux with systemd, it's convenient to use systemd timers. For macOS systems, we can use built-in LaunchAgents. For Windows we can use ScheduledTasks. Any OS having something cron-like will also work! Here follows a step-by step tutorial on how to set it up, with my sample script and configurations that you can modify to suit your needs. -Note, you can use any restic's supported [storage backends](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html). The setup should be similar but you will have to use other configuration variables to match your backend of choice. +Note, you can use any restic's supported [storage backends](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html). The setup should be similar, but you will have to use other configuration variables to match your backend of choice. ## Project Scope The scope for this is not to be a full-fledged super solution that solves all the problems and all possible setups. The aim is to be a hackable code base for you to start sewing up the perfect backup solution that fits your requirements! -Nevertheless the project should work out of the box, be minimal but still open the doors for configuration and extensions by users. +Nevertheless, the project should work out of the box, be minimal but still open the doors for configuration and extensions by users. To use a different storage backend than B2, you should only need to tweak a few settings variables in the backup profile as well as some restic arguments inside `restic_backup.sh`. @@ -442,12 +442,18 @@ To create a different backup and use you can do: # restic_backup.sh ``` +### Optional: Summary stats log + +When enabled, it will write to a CSV log file the stats after each backup. Can be enabled by uncommenting its env variable (`RESTIC_BACKUP_STATS_DIR`) on the global environment file or defining it on a specific profile. + +The stats log (as well as) the desktop notifications incur in an additional run of `restic snapshots` and `restic diff`. This execution is shared with the notifications (no extra run). + ### Optional: Desktop Notifications -It's a good idea to be on top of your backups to make sure that they don't increase a lot in size and incur high costs. However it's notoriously tricky to make GUI notifications correctly from a non-user process (e.g. root). +It's a good idea to be on top of your backups to make sure that they don't increase a lot in size and incur high costs. However, it's notoriously tricky to make GUI notifications correctly from a non-user process (e.g. root). -Therefore this project provides a lightweight solution for desktop notifications that works like this: Basically `restic_backup.sh` will append a summary line of the last backup to a user-owned file (the user running your OS's desktop environment) in a fire-and-forget fashion. Then the user has a process that reads this and forward each line as a new message to the desktop environment in use. +Therefore, this project provides a lightweight solution for desktop notifications that works like this: Basically `restic_backup.sh` will append a summary line of the last backup to a user-owned file (the user running your OS's desktop environment) in a fire-and-forget fashion. Then the user has a process that reads this and forward each line as a new message to the desktop environment in use. To set desktop notifications up: 1. Create a special FIFO file as your desktop user: @@ -456,12 +462,11 @@ To set desktop notifications up: ``` 1. In your profile, e.g. `/etc/restic/default.sh`, set: ```bash - RESTIC_NOTIFY_BACKUP_STATS=true RESTIC_BACKUP_NOTIFICATION_FILE=/home/user/.cache/notification-queue ``` 1. Create a listener on the notification queue file that forwards to desktop notifications * Linux auto start + cross-platform notifier / notify-send - * [notification-queue-notifier](https://github.com/gerardbosch/dotfiles/blob/ddc1491056822eab45dedd131f1946308ef62135/home/bin/notification-queue-notifier) + * [notification-queue-notifier](https://github.com/gerardbosch/dotfiles/blob/2130d54daa827e7f885abac0d4f10b6f67d28ad3/home/bin/notification-queue-notifier) * [notification-queue.desktop](https://github.com/gerardbosch/dotfiles-linux/blob/ea0f75bfd7a356945544ecaa42a2fc35c9fab3a1/home/.config/autostart/notification-queue.desktop) * macOS auto start + [terminal-notifier](https://github.com/julienXX/terminal-notifier) * [notification-queue-notifier.sh](https://github.com/erikw/dotfiles/blob/8a942defe268292200b614951cdf433ddccf7170/bin/notification-queue-notifier.sh) @@ -554,7 +559,7 @@ $ source /etc/restic/default.env.sh $ bash -x /bin/restic_backup.sh ``` -To debug smaller portions of of the backup script, insert these lines at the top and bottom of the relevant code portions e.g.: +To debug smaller portions of the backup script, insert these lines at the top and bottom of the relevant code portions e.g.: ```bash set -x diff --git a/bin/restic_backup.sh b/bin/restic_backup.sh index 1ad5d57..279ed52 100755 --- a/bin/restic_backup.sh +++ b/bin/restic_backup.sh @@ -49,6 +49,30 @@ warn_on_missing_envvars() { fi } +# Log the backup summary stats to a CSV file +logBackupStatsCsv() { + local snapId="$1" added="$2" removed="$3" snapSize="$4" + local logFile + logFile="${RESTIC_BACKUP_STATS_DIR}/$(date '+%Y')-stats.log.csv" + test -e "$logFile" || install -D -m 0644 <(echo "Date, Snapshot ID, Added, Removed, Snapshot size") "$logFile" + # DEV-NOTE: using `ex` due `sed` inconsistencies (GNU vs. BSD) and `awk` cannot edit in-place. `ex` does a good job + printf '1a\n%s\n.\nwq\n' "$(date '+%F %H:%M:%S'), ${snapId}, ${added}, ${removed}, ${snapSize}" | ex "$logFile" +} + +# Notify the backup summary stats to the user +notifyBackupStats() { + local statsMsg="$1" + if [ -w "$RESTIC_BACKUP_NOTIFICATION_FILE" ]; then + echo "$statsMsg" >> "$RESTIC_BACKUP_NOTIFICATION_FILE" + else + echo "[WARN] Couldn't write to the backup notification file. File not found or not writable: ${RESTIC_BACKUP_NOTIFICATION_FILE}" + fi +} + +# ------------ +# === Main === +# ------------ + assert_envvars \ RESTIC_BACKUP_PATHS RESTIC_BACKUP_TAG \ RESTIC_BACKUP_EXCLUDE_FILE RESTIC_BACKUP_EXTRA_ARGS RESTIC_REPOSITORY RESTIC_VERBOSITY_LEVEL \ @@ -138,23 +162,22 @@ wait $! echo "Backup & cleaning is done." -# (optionally) Notify about backup summary stats. -if [ "$RESTIC_NOTIFY_BACKUP_STATS" = true ]; then - if [ -w "$RESTIC_BACKUP_NOTIFICATION_FILE" ]; then - echo 'Notifications are enabled: Silently computing backup summary stats...' - - snapshot_size=$(restic stats latest --tag "$RESTIC_BACKUP_TAG" | grep -i 'total size:' | cut -d ':' -f2 | xargs) # xargs acts as trim - latest_snapshot_diff=$(restic snapshots --tag "$RESTIC_BACKUP_TAG" --latest 2 --compact \ - | grep -Ei "^[abcdef0-9]{8} " \ - | awk '{print $1}' \ - | tail -2 \ - | tr '\n' ' ' \ - | xargs restic diff) - added=$(echo "$latest_snapshot_diff" | grep -i 'added:' | awk '{print $2 " " $3}') - removed=$(echo "$latest_snapshot_diff" | grep -i 'removed:' | awk '{print $2 " " $3}') - - echo "Added: ${added}. Removed: ${removed}. Snap size: ${snapshot_size}" >> "$RESTIC_BACKUP_NOTIFICATION_FILE" - else - echo "[WARN] Couldn't write the backup summary stats. File not found or not writable: ${RESTIC_BACKUP_NOTIFICATION_FILE}" - fi +# (optional) Compute backup summary stats +if [[ -n "$RESTIC_BACKUP_STATS_DIR" || -n "$RESTIC_BACKUP_NOTIFICATION_FILE" ]]; then + echo 'Silently computing backup summary stats...' + latest_snapshots=$(restic snapshots --tag "$RESTIC_BACKUP_TAG" --latest 2 --compact \ + | grep -Ei "^[abcdef0-9]{8} " \ + | awk '{print $1}' \ + | tail -2 \ + | tr '\n' ' ') + latest_snapshot_diff=$(echo "$latest_snapshots" | xargs restic diff) + added=$(echo "$latest_snapshot_diff" | grep -i 'added:' | awk '{print $2 " " $3}') + removed=$(echo "$latest_snapshot_diff" | grep -i 'removed:' | awk '{print $2 " " $3}') + snapshot_size=$(restic stats latest --tag "$RESTIC_BACKUP_TAG" | grep -i 'total size:' | cut -d ':' -f2 | xargs) # xargs acts as trim + snapshotId=$(echo "$latest_snapshots" | cut -d ' ' -f2) + statsMsg="Added: ${added}. Removed: ${removed}. Snap size: ${snapshot_size}" + + echo "$statsMsg" + test -n "$RESTIC_BACKUP_STATS_DIR" && logBackupStatsCsv "$snapshotId" "$added" "$removed" "$snapshot_size" + test -n "$RESTIC_BACKUP_NOTIFICATION_FILE" && notifyBackupStats "$statsMsg" fi diff --git a/etc/restic/_global.env.sh b/etc/restic/_global.env.sh index 8694320..390cd61 100644 --- a/etc/restic/_global.env.sh +++ b/etc/restic/_global.env.sh @@ -1,6 +1,6 @@ # shellcheck shell=sh -# Global envionment variables +# Global environment variables # These variables are sourced FIRST, and any values inside of *.env.sh files for # specific configurations will override if also defined there. @@ -30,6 +30,8 @@ export RESTIC_BACKUP_EXTRA_ARGS= # Override this value in a profile if needed. export RESTIC_VERBOSITY_LEVEL=0 -# (optional) Desktop notifications. See restic_backup.sh for details on how to set this up. -export RESTIC_NOTIFY_BACKUP_STATS=false +# (optional, uncomment to enable) Backup summary stats log: snapshot size, etc. (empty/unset won't log) +#export RESTIC_BACKUP_STATS_DIR="$INSTALL_PREFIX/var/log/restic-automatic-backup-scheduler" + +# (optional) Desktop notifications. See README and restic_backup.sh for details on how to set this up (empty/unset means disabled) export RESTIC_BACKUP_NOTIFICATION_FILE=