Skip to content

Commit

Permalink
feat: Log the backup stats summary to a file (#128)
Browse files Browse the repository at this point in the history 
* docs: Fix typos and minor improvements

* feat: Log the backup stats summary to a CSV file

The log records the added, removed and snapshot size after each backup.

* feat: Add the snapshot ID to the stats log

* docs: Update README with the stats log info

* fix: Linter 1: Quote var

* fix: Linter 2: Shellcheck declare and assign separately

* feat: Turn the stats log into an opt-in
  • Loading branch information
@gerardbosch
gerardbosch authored Dec 12, 2023
1 parent e681b8f commit 69e21d5
Show file tree
Hide file tree
Showing 3 changed files with 60 additions and 30 deletions.
21 changes: 13 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,16 @@
# Intro
[restic](https://restic.net/) is a command-line tool for making backups, the right way. Check the official website for a feature explanation. As a storage backend, I recommend [Backblaze B2](https://www.backblaze.com/b2/cloud-storage.html) as restic works well with it, and it is (at the time of writing) very affordable for the hobbyist hacker! (anecdotal: I pay for my full-systems backups each month typically < 1 USD).

Unfortunately restic does not come pre-configured with a way to run automated backups, say every day. However it's possible to set this up yourself using built-in tools in your OS and some wrappers. For Linux with systemd, it's convenient to use systemd timers. For macOS systems, we can use built-in LaunchAgents. For Windows we can use ScheduledTasks. Any OS having something cron-like will also work!
Unfortunately restic does not come pre-configured with a way to run automated backups, say every day. However, it's possible to set this up yourself using built-in tools in your OS and some wrappers. For Linux with systemd, it's convenient to use systemd timers. For macOS systems, we can use built-in LaunchAgents. For Windows we can use ScheduledTasks. Any OS having something cron-like will also work!

Here follows a step-by step tutorial on how to set it up, with my sample script and configurations that you can modify to suit your needs.

Note, you can use any restic's supported [storage backends](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html). The setup should be similar but you will have to use other configuration variables to match your backend of choice.
Note, you can use any restic's supported [storage backends](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html). The setup should be similar, but you will have to use other configuration variables to match your backend of choice.

## Project Scope
The scope for this is not to be a full-fledged super solution that solves all the problems and all possible setups. The aim is to be a hackable code base for you to start sewing up the perfect backup solution that fits your requirements!

Nevertheless the project should work out of the box, be minimal but still open the doors for configuration and extensions by users.
Nevertheless, the project should work out of the box, be minimal but still open the doors for configuration and extensions by users.

To use a different storage backend than B2, you should only need to tweak a few settings variables in the backup profile as well as some restic arguments inside `restic_backup.sh`.

Expand Down Expand Up @@ -442,12 +442,18 @@ To create a different backup and use you can do:
# restic_backup.sh
```

### Optional: Summary stats log

When enabled, it will write to a CSV log file the stats after each backup. Can be enabled by uncommenting its env variable (`RESTIC_BACKUP_STATS_DIR`) on the global environment file or defining it on a specific profile.

The stats log (as well as) the desktop notifications incur in an additional run of `restic snapshots` and `restic diff`. This execution is shared with the notifications (no extra run).

### Optional: Desktop Notifications
<img src="img/macos_notification.png" align="right" />

It's a good idea to be on top of your backups to make sure that they don't increase a lot in size and incur high costs. However it's notoriously tricky to make GUI notifications correctly from a non-user process (e.g. root).
It's a good idea to be on top of your backups to make sure that they don't increase a lot in size and incur high costs. However, it's notoriously tricky to make GUI notifications correctly from a non-user process (e.g. root).

Therefore this project provides a lightweight solution for desktop notifications that works like this: Basically `restic_backup.sh` will append a summary line of the last backup to a user-owned file (the user running your OS's desktop environment) in a fire-and-forget fashion. Then the user has a process that reads this and forward each line as a new message to the desktop environment in use.
Therefore, this project provides a lightweight solution for desktop notifications that works like this: Basically `restic_backup.sh` will append a summary line of the last backup to a user-owned file (the user running your OS's desktop environment) in a fire-and-forget fashion. Then the user has a process that reads this and forward each line as a new message to the desktop environment in use.

To set desktop notifications up:
1. Create a special FIFO file as your desktop user:
Expand All @@ -456,12 +462,11 @@ To set desktop notifications up:
```
1. In your profile, e.g. `/etc/restic/default.sh`, set:
```bash
RESTIC_NOTIFY_BACKUP_STATS=true
RESTIC_BACKUP_NOTIFICATION_FILE=/home/user/.cache/notification-queue
```
1. Create a listener on the notification queue file that forwards to desktop notifications
* Linux auto start + cross-platform notifier / notify-send
* [notification-queue-notifier](https://github.com/gerardbosch/dotfiles/blob/ddc1491056822eab45dedd131f1946308ef62135/home/bin/notification-queue-notifier)
* [notification-queue-notifier](https://github.com/gerardbosch/dotfiles/blob/2130d54daa827e7f885abac0d4f10b6f67d28ad3/home/bin/notification-queue-notifier)
* [notification-queue.desktop](https://github.com/gerardbosch/dotfiles-linux/blob/ea0f75bfd7a356945544ecaa42a2fc35c9fab3a1/home/.config/autostart/notification-queue.desktop)
* macOS auto start + [terminal-notifier](https://github.com/julienXX/terminal-notifier)
* [notification-queue-notifier.sh](https://github.com/erikw/dotfiles/blob/8a942defe268292200b614951cdf433ddccf7170/bin/notification-queue-notifier.sh)
Expand Down Expand Up @@ -554,7 +559,7 @@ $ source /etc/restic/default.env.sh
$ bash -x /bin/restic_backup.sh
```

To debug smaller portions of of the backup script, insert these lines at the top and bottom of the relevant code portions e.g.:
To debug smaller portions of the backup script, insert these lines at the top and bottom of the relevant code portions e.g.:

```bash
set -x
Expand Down
61 changes: 42 additions & 19 deletions bin/restic_backup.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,30 @@ warn_on_missing_envvars() {
fi
}

# Log the backup summary stats to a CSV file
logBackupStatsCsv() {
local snapId="$1" added="$2" removed="$3" snapSize="$4"
local logFile
logFile="${RESTIC_BACKUP_STATS_DIR}/$(date '+%Y')-stats.log.csv"
test -e "$logFile" || install -D -m 0644 <(echo "Date, Snapshot ID, Added, Removed, Snapshot size") "$logFile"
# DEV-NOTE: using `ex` due `sed` inconsistencies (GNU vs. BSD) and `awk` cannot edit in-place. `ex` does a good job
printf '1a\n%s\n.\nwq\n' "$(date '+%F %H:%M:%S'), ${snapId}, ${added}, ${removed}, ${snapSize}" | ex "$logFile"
}

# Notify the backup summary stats to the user
notifyBackupStats() {
local statsMsg="$1"
if [ -w "$RESTIC_BACKUP_NOTIFICATION_FILE" ]; then
echo "$statsMsg" >> "$RESTIC_BACKUP_NOTIFICATION_FILE"
else
echo "[WARN] Couldn't write to the backup notification file. File not found or not writable: ${RESTIC_BACKUP_NOTIFICATION_FILE}"
fi
}

# ------------
# === Main ===
# ------------

assert_envvars \
RESTIC_BACKUP_PATHS RESTIC_BACKUP_TAG \
RESTIC_BACKUP_EXCLUDE_FILE RESTIC_BACKUP_EXTRA_ARGS RESTIC_REPOSITORY RESTIC_VERBOSITY_LEVEL \
Expand Down Expand Up @@ -138,23 +162,22 @@ wait $!

echo "Backup & cleaning is done."

# (optionally) Notify about backup summary stats.
if [ "$RESTIC_NOTIFY_BACKUP_STATS" = true ]; then
if [ -w "$RESTIC_BACKUP_NOTIFICATION_FILE" ]; then
echo 'Notifications are enabled: Silently computing backup summary stats...'

snapshot_size=$(restic stats latest --tag "$RESTIC_BACKUP_TAG" | grep -i 'total size:' | cut -d ':' -f2 | xargs) # xargs acts as trim
latest_snapshot_diff=$(restic snapshots --tag "$RESTIC_BACKUP_TAG" --latest 2 --compact \
| grep -Ei "^[abcdef0-9]{8} " \
| awk '{print $1}' \
| tail -2 \
| tr '\n' ' ' \
| xargs restic diff)
added=$(echo "$latest_snapshot_diff" | grep -i 'added:' | awk '{print $2 " " $3}')
removed=$(echo "$latest_snapshot_diff" | grep -i 'removed:' | awk '{print $2 " " $3}')

echo "Added: ${added}. Removed: ${removed}. Snap size: ${snapshot_size}" >> "$RESTIC_BACKUP_NOTIFICATION_FILE"
else
echo "[WARN] Couldn't write the backup summary stats. File not found or not writable: ${RESTIC_BACKUP_NOTIFICATION_FILE}"
fi
# (optional) Compute backup summary stats
if [[ -n "$RESTIC_BACKUP_STATS_DIR" || -n "$RESTIC_BACKUP_NOTIFICATION_FILE" ]]; then
echo 'Silently computing backup summary stats...'
latest_snapshots=$(restic snapshots --tag "$RESTIC_BACKUP_TAG" --latest 2 --compact \
| grep -Ei "^[abcdef0-9]{8} " \
| awk '{print $1}' \
| tail -2 \
| tr '\n' ' ')
latest_snapshot_diff=$(echo "$latest_snapshots" | xargs restic diff)
added=$(echo "$latest_snapshot_diff" | grep -i 'added:' | awk '{print $2 " " $3}')
removed=$(echo "$latest_snapshot_diff" | grep -i 'removed:' | awk '{print $2 " " $3}')
snapshot_size=$(restic stats latest --tag "$RESTIC_BACKUP_TAG" | grep -i 'total size:' | cut -d ':' -f2 | xargs) # xargs acts as trim
snapshotId=$(echo "$latest_snapshots" | cut -d ' ' -f2)
statsMsg="Added: ${added}. Removed: ${removed}. Snap size: ${snapshot_size}"

echo "$statsMsg"
test -n "$RESTIC_BACKUP_STATS_DIR" && logBackupStatsCsv "$snapshotId" "$added" "$removed" "$snapshot_size"
test -n "$RESTIC_BACKUP_NOTIFICATION_FILE" && notifyBackupStats "$statsMsg"
fi
8 changes: 5 additions & 3 deletions etc/restic/_global.env.sh
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# shellcheck shell=sh

# Global envionment variables
# Global environment variables
# These variables are sourced FIRST, and any values inside of *.env.sh files for
# specific configurations will override if also defined there.

Expand Down Expand Up @@ -30,6 +30,8 @@ export RESTIC_BACKUP_EXTRA_ARGS=
# Override this value in a profile if needed.
export RESTIC_VERBOSITY_LEVEL=0

# (optional) Desktop notifications. See restic_backup.sh for details on how to set this up.
export RESTIC_NOTIFY_BACKUP_STATS=false
# (optional, uncomment to enable) Backup summary stats log: snapshot size, etc. (empty/unset won't log)
#export RESTIC_BACKUP_STATS_DIR="$INSTALL_PREFIX/var/log/restic-automatic-backup-scheduler"

# (optional) Desktop notifications. See README and restic_backup.sh for details on how to set this up (empty/unset means disabled)
export RESTIC_BACKUP_NOTIFICATION_FILE=

0 comments on commit 69e21d5

Please sign in to comment.