Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not passing --forget to 'repair snapshots' #365

Open
davad opened this issue Jun 4, 2024 · 1 comment
Open

Not passing --forget to 'repair snapshots' #365

davad opened this issue Jun 4, 2024 · 1 comment
Assignees

Comments

@davad
Copy link

davad commented Jun 4, 2024

I think there's a bug with repair snapshots.

I have missing data in my B2-backed restic repo. I'm ok losing the data, but I need to repair snapshots so I can prune again. I'm following the 'Troubleshooting' steps in Restic's documentation.

Reproduction steps:

  1. Have missing blobs in your repo
  2. Repair the index so the repo knows the blobs are missing: resticprofile -n <profile> repair index
  3. Remove missing data from snapshots: resticprofile -n <profile> repair snapshot --forget
  4. Run resticprofile -n <profile> check

Expected behavior

Restic repairs any snapshots with missing data and drops the old, damaged snapshot. Running check results in the message 'no errors were found'

Actual behavior

Restic repairs any snapshots with missing data and retains the old, damaged snapshot. Running check reported repository errors.

Version info

% restic version
restic 0.16.4 compiled with go1.21.6 on linux/amd64

% resticprofile version
resticprofile version 0.26.0 commit 2ab1f3a38467114aeb4095c0924817b369280e39

Investigation

It seemed like Restic was ignoring the --forget flag. I ran the repair again and looked at the processes running. I also tried adding a repair-snapshots section and setting forget: true there. Both times when I ran ps aux | grep restic, I saw this:

resticprofile -n remote repair snapshots
/usr/bin/restic repair --password-file=<snip> --repo=b2:<snip> snapshots

I thought it was weird that --forget didn't show up there. So I ran restic directly and checked the running processes:

restic repair --password-file=<snip> --repo=b2:<snip> snapshots --forget

After running it directly, I was able to run resticprofile -n <profile> check with no errors reported.

Summary

It looks like resticprofile repair snapshots is dropping the --forget parameter. It also seems to be ignoring the config option for profile.repair-snapshots -> forget: true.

@davad
Copy link
Author

davad commented Jun 4, 2024

I was able to create a dummy repo to reproduce the issue. Here are the steps I followed.

Create repo

  1. Minimal resticprofile config
test:
  password-file: "{{ .ConfigDir }}/{{ .Profile.Name }}.key"
  repository: /path/to/restic-repo-test
  lock: "{{ .ConfigDir }}/resticprofile-profile-{{ .Profile.Name }}.lock"
  force-inactive-lock: true
  backup:
    source: "/path/to/test-restic-source"
    check-before: false
    run-finally: 'mv {{ tempFile "backup.log" }} {{ .ConfigDir }}/{{ .Profile.Name }}-backup{{ .Now.Format "2006-01-02T15-04-05" }}.log'
  1. Generate key:
resticprofile -n test generate --random-key > ~/.config/resticprofile/test.key
  1. Prep source and destination
mkdir -p /path/to/restic-repo-test
mkdir -p /path/to/test-restic-source
  1. Initialize repo
resticprofile -n test init

Generate data and backup

  1. Generate a random 100MiB file
dd if=/dev/urandom of=/path/to/test-restic-source/random status=progress bs=1M count=100
  1. Run backup
resticprofile -n test backup

Corrupt backup

  1. Find a random piece of data and delete it
cd /path/to/restic-repo-test/data
find ./ -type f

# ./05/053c4c9c037f38b94bbf85dd6800b91042fcb5b2336a8007bd3f440143df4ac8
# ./8b/8b68097fb3b9327e213e138482519602a900efae818b1b981533f96dc14d87b0
# ./f7/f7d1e3b543b9713b493c5a9cacaa944d7cb123a8b2423a327501fddbd48320b0
# ./38/38907e30e29fa06b468110399d7e0d22396cf2202540b80a7583dec92c6eff9c
# ./cc/ccf9cd6edd5e3c1643bd12dbd587d2b7405ce0b651a755e902e22372dc7ffa16
# ./2e/2edb424bd3c6e4a9a8c189446d1cc338efbb21603ccb1a4447ae4545a5d63a76
# ./52/52e20a63c8a508fdc8252ab774ac37075992f19d355ffa79445484682673338a

rm ./05/053c4c9c037f38b94bbf85dd6800b91042fcb5b2336a8007bd3f440143df4ac8
  1. Verify the repo knows it's corrupted
resticprofile -n test check

# ...
# check all packs
# check snapshots, trees and blobs
# pack 053c4c9c037f38b94bbf85dd6800b91042fcb5b2336a8007bd3f440143df4ac8: does not exist
# [0:00] 100.00%  1 / 1 snapshots
# Fatal: repository contains errors

Recovery

  1. Repair index, so the indices don't reference missing data
resticprofile -n test repair index
  1. Repair snapshots to drop missing data
resticprofile -n test repair snapshots --forget
  1. Run check to verify repo integrity
resticprofile -n test check

Actual output

When running repair snapshots --forget, we see output like this:

% resticprofile -n test repair snapshots                                                                                                                                                                                                         

# 2024/06/04 16:01:13 profile 'test': starting 'repair'
# repository 9a779c38 opened (version 2, compression level auto)
# [0:00] 100.00%  1 / 1 index files loaded
# 
# snapshot fd4e3a1a of [/path/to/test-restic-source] at 2024-06-04 15:58:44.55902647 -0400 EDT)
#   file "/path/to/test-restic-source/random3": removed missing content
# saved new snapshot 5e00ec93
# 
# modified 1 snapshots
# 2024/06/04 16:01:14 profile 'test': finished 'repair'

If we run repair snapshots --forget after that, the output looks like this:

% restic repair --password-file=~/.config/resticprofile/test.key --repo=/path/to/restic-repo-test snapshots --forget

# repository 9a779c38 opened (version 2, compression level auto)
# [0:00] 100.00%  2 / 2 index files loaded
# 
# snapshot fd4e3a1a of [/path/to/test-restic-source] at 2024-06-04 15:58:44.55902647 -0400 EDT)
#   file "/path/to/test-restic-source/random3": removed missing content
# saved new snapshot 39ea6ade
# removed old snapshot fd4e3a1a
# 
# snapshot 5e00ec93 of [/path/to/test-restic-source] at 2024-06-04 15:58:44.55902647 -0400 EDT)
# 
# modified 1 snapshots

@jkellerer jkellerer self-assigned this Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants