restic
Don't print errors when reading or removing lock files
Output of restic version
restic 0.17.3 compiled with go1.23.3 on windows/amd64
What backend/service did you use to store the repository?
pCloud via their desktop app (mounted to P:)
Problem description / Steps to reproduce
Hello. I have a backup script I created that does 100+ backups in a row for each of my games' save files. The script runs up to 10 of these backup processes concurrently with --retry-lock to save time just when spawning the processes. I use --quiet on all these processes, and despite restic correctly utilizing the lock files to only allow one backup at a time, I get several warnings in the console of a lock file failing to be read or removed because it no longer exists or is being used by another process.
Expected behavior
Restic should not print these errors, since it seems to immediately retry without any issue.
Actual behavior
Load(<lock/ef7fe36487>, 0, 0) failed: open \\?\P:\ludusavi-restic\locks\ef7fe36487e7907dd83100c985eaf5da005daeb9b0bb765a4e979e99838a3b44: The system cannot find the file specified.
Load(<lock/ef7fe36487>, 0, 0) failed: open \\?\P:\ludusavi-restic\locks\ef7fe36487e7907dd83100c985eaf5da005daeb9b0bb765a4e979e99838a3b44: The system cannot find the file specified.
Remove(<lock/f33b65260f>) returned error, retrying after 948.956403ms: remove \\?\P:\ludusavi-restic\locks\f33b65260f81bfa4d13377250f5d2f5bf06a90238620aa5cfa6e057c34e0beac: The process cannot access the file because it is being used by another process.
Remove(<lock/f33b65260f>) operation successful after 1 retries
Do you have any idea what may have caused this?
Concurrency between multiple processes.
Did restic help you today? Did it make you happy in any way?
From a performance point of view it would be much more efficient to just keep all save files in a single snapshot. But the unnecessary warning are still a (tiny) problem though.
Load(<lock/ef7fe36487>, 0, 0) failed: open \?\P:\ludusavi-restic\locks\ef7fe36487e7907dd83100c985eaf5da005daeb9b0bb765a4e979e99838a3b44: The system cannot find the file specified.
It's somewhat odd that the error for this file shows up twice. Apparently it takes some time for the file to also disappear from the file listing. But we can definitely do a better job of filtering those errors, we'll just need some way to ensure that continuously inaccessible lock files or other errors are still logged.
Remove(<lock/f33b65260f>) returned error, retrying after 948.956403ms: remove \?\P:\ludusavi-restic\locks\f33b65260f81bfa4d13377250f5d2f5bf06a90238620aa5cfa6e057c34e0beac: The process cannot access the file because it is being used by another process.
That is a Windows-specific error. Fixing this would require the local backend to use a custom function on Windows to open the file such it can be marked for deletion even while some other process has opened the file.
It's worth noting that I just had this exact issue cause an error with Backrest, which parses the data from --json:
error: failed to get snapshots for plan: get snapshots for plan "MinecraftJava": command "C:\\Users\\ethan\\scoop\\shims\\restic.exe snapshots --json -o sftp.args=-oBatchMode=yes --tag plan:Minec..." failed: command output is not valid JSON: invalid character 'L' looking for beginning of value
Output:
Load(<lock/535a55f94b>, 0, 0) failed: <lock/535a55f94b> does not exist
Load(<lock/535a55f94b>, 0, 0) failed: <lock/535a55f94b> does not exist
[{"time":"2025-03-20T23:26:34.1128646-06:00","tree":"dacca9d05aebd70c55ab4d7ad2e8ad8ca6e3065bdbb0b55f5b39977346a68c4e","paths":["C:\\Users\\ethan\\AppData\\Roaming\\.minecraft"],"hostname":"Ethan-PC","username":"ETHAN-PC\\ethan","excludes":[".minecraft/assets",".minecraft/libraries",".minecraft/jvm",".minecraft/versions",".minecraft/webcache2"],"tags":["plan:MinecraftJava","created-by:ethan-pc"],"program_version":"restic 0.17.3","summary":{"backup_start":"2025-03-20T23:26:34.1128646-06:00","backup_end":"2025-03-20T23:30:38.817514-06:00","files_new":11114,"files_changed":0,"files_unmodified":0,"dirs_new":773,"dirs_changed":0,"dirs_unmodified":0,"data_blobs":5068,"tree_blobs":725,"data_added":1467126818,"data_added_packed":1179494932,"total_files_processed":11114,"total_bytes_processed":1991047193},"id":"306bba2261deb16c82e07bb66049051e11724c8dcd59
... 46 bytes truncated ...
Before you backup your saves, did you try that using the save-off command in Minecraft server session?
...my backup plan was for my Minecraft client, which wasn't running at the time of backup.