[Feature request] backup: select a group of snapshots

[fvsch]

Currently, using rustic backup with a TOML profile which defines several snapshots, allows:

1. Backing up all the snapshots.
2. Backing up one or several specific snapshot(s) with --name <snapshot_name>.

It would be great to be able to select a predefined group of snapshots, instead of having to list N named snapshots with --name A --name B --name C ....

Example snapshots setup

My use case is that I have several drives (or volumes), some always mounted (laptop’s system volume), and some often not (external SSD).

I would like to be able to back up all the snapshots living on one volume, while ignoring others. This is possible today with rustic, but not very ergonomic.

Let’s say we have the following configured snapshots:

name	label	tags	sources
Alpha	Alpha	local	/home/me/Alpha
Bravo	Bravo	local	/home/me/Bravo
Charlie	Charlie	local	/home/me/Charlie
Delta	Delta	external	/mnt/mydrive/Delta
Echo	Echo	external	/mnt/mydrive/Echo
Foxtrot	Foxtrot	external	/mnt/mydrive/Foxtrot

How can we address the use case where the first 3 snapshots have accessible sources, but the other 3 do not because the external drive is not mounted (e.g. using my laptop on the go)?

Workaround 0: just ignore the errors

Without any selection or filtering, rustic attempts to back up all configured snapshots. This will print errors for missing sources, but snapshots with accessible sources will be backed up correctly.

Some errors will be printed:

> rustic backup
...
[ERROR] error backing up /mnt/mydrive/Delta: error sanitizing source=s"PathList(["/mnt/mydrive/Delta"])"
...
[ERROR] error backing up /mnt/mydrive/Echo: error sanitizing source=s"PathList(["/mnt/mydrive/Echo"])"
...
[ERROR] error backing up /mnt/mydrive/Foxtrot: error sanitizing source=s"PathList(["/mnt/mydrive/Foxtrot"])"
...
error: Not all snapshots were generated successfully!

In practice, this works. But it requires training oneself to ignore specific errors while not ignoring different errors, which is a usability problem and can lead to mistakes.

Workaround 1: list all desired snapshots by name

rustic backup --name Alpha --name Bravo --name Charlie

This works, but it partly defeats the purpose of having configuration files for snapshots, and can be error-prone when there are many names to list.

Workaround 2: one snapshot per volume

This more extreme solution is to have just one named snapshot per volume:

name	label	tags	sources
local	local	-	/home/me/Alpha, /home/me/Bravo, /home/me/Charlie
external	external	-	/mnt/mydrive/Delta, /mnt/mydrive/Echo, /mnt/mydrive/Foxtrot

This allows using a single name to back up many sources:

# back up all configured snapshots
rustic backup

# back up local drive snapshots
rustic backup --name local

# back up external ssd snapshots
rustic backup --name external

Downsides:

• Losing the ability to not create new snapshots for data which does not change often. E.g. if “Alpha” gets changes hourly, but “Charlie” gets changes every other week, grouping them has some ergonomic downsides when using rustic snapshots, rustic diff and rustic forget IMHO (though I’m a new user of rustic, so maybe I’m holding it wrong!).
• Readability: snapshots often get reported in terminal output as a list of paths (rather than by name or label), so having a snapshot configured with 3 or 10 paths can become unwieldy very fast.

Workaround 3: select snapshots with use-profiles

This is what I’m doing currently. My ~/.config/rustic directory looks like (abridged):

.config/rustic
├── sources
│   ├── external.toml
│   └── local.toml
├── repos
│   └── sbox.toml
├── all.toml (use-profiles = ["repos/sbox", "sources/local", "sources/external"])
└── local.toml (use-profiles = ["repos/sbox", "sources/local"])

The snapshots are defined in sources/local.toml (for local files) and sources/external.toml (for the external SSD files).

This way, I can run either:

• rustic -P all backup
• rustic -P local backup

Downsides:

• It’s a bit hard to set up.
• You have to remember to use the correct profile for backup.
• The profile you select doesn’t matter for other commands (e.g. rustic -P local snapshots and rustic -P all snapshots do the same thing), which doesn’t help forming the muscle memory for using the correct profile.
• This example has one repository and 2 sets of snapshots, so it needs 2 profiles as entry points. If you have 2 repositories and 3 sets of snapshots, you need 6 entry points. The combinatorics can break down quickly.

Feature request: filter by tags or paths

I tried using --filter-paths and --filter-tags, because the output of rustic backup --help seemed to suggest that those options are available with rustic backup, but they seem ignored (a bug in the help output?).

It would be cool if they did work, or if similar options specific to rustic backup were available:

# back up all configured snapshots
rustic backup

# back up local drive snapshots
rustic backup --filter-paths /home/me/
# or
rustic backup --filter-tags local

# back up external ssd snapshots
rustic backup --filter-paths /mnt/mydrive/
# or
rustic backup --filter-tags external

If these (or similar options not using the word filter, e.g. --select-tags or --only-tags) are a no-go because tags are not supposed to be used to select snapshot configs, maybe adding a way to configure lists of snapshot names could work? Something like:

[backup.groups]
local = ["Alpha", "Bravo", "Charlie"]
external = ["Delta", "Echo", "Foxtrot"]

To be used as:

rustic backup --group local

(Bikeshedding: possibly confusing to use “group”, since --group-by exists. Possible synonyms include “set” and “batch”.)

Personally I like filtering by paths or tags best, because it uses the existing metadata for snapshots. I’m already not sure why rustic has both label and name in snapshot metadata, when using the labels to select specific snapshots may have sufficed. Maybe there was an intent to keep repository metadata separate from the ways to select snapshots to back up?

[aawsome]

Hi @fvsch!

Actually, there is some similar solution as you requested with --filter-paths already present:
If you run rustic backup <PATH> and that path is defined for a snapshot in the toml profile, the options from the profile are taken.
However, this only works with a complete path list (rustic backup <PATH1> <PATH2> will look for a snapshot backing with sources PATH1, PATH2 and not create 2 snapshots for <PATH1>, <PATH2> even if they would exist)

I thought a bit about this feature request. And I think you are right. Applying the -filter-* options also to snapshot configs from the toml profile is quite natural and IMO doesn't bring in too much unexpected outcome. But I'll need to sort this out a bit and this would also need a bit of refactoring. So this may take a bit to get implemented..