Restore of UNC Paths within windows failing (reproducible)

[vessev]

Output of restic version

restic 0.9.6 compiled with go1.13.4 on windows/amd64

How did you run restic exactly?

Open elevated Powershell

> C:\restic_0.9.6_w\restic.exe version restic 0.9.6 compiled with go1.13.4 on windows/amd64

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.3471
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.3471
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

> $env:RESTIC_PASSWORD = 'restic'

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test init created restic repository 562ab4bb33 at K:\ResticBackup\test

Please note that knowledge of your password is required to access the repository. Losing your password means that your data is irrecoverably lost.

> ls "\\10.16.10.220\Test Folder"

    Directory: \\10.16.10.220\Test Folder


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       25.06.2020     00:49                Folder 1
d-----       25.06.2020     00:49                Folder 2

> ls "\\10.16.10.220\TestFolderNoSpace"

    Directory: \\10.16.10.220\TestFolderNoSpace


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       25.06.2020     00:49                Folder 2
d-----       25.06.2020     00:49                Folder 1

> ls "K:\TestFolderNoSpace"

    Directory: K:\TestFolderNoSpace


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       25.06.2020     00:49                Folder 2
d-----       25.06.2020     00:49                Folder 1

> ls "K:\Test Folder"

    Directory: K:\Test Folder


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       25.06.2020     00:49                Folder 1
d-----       25.06.2020     00:49                Folder 2

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "\\10.16.10.220\Test Folder" repository 562ab4bb opened successfully, password is correct Fatal: unable to save snapshot: Lstat: CreateFile \?\UNC\10.16.10.220\Test Folder\10.16.10.220: The system cannot find the file specified.

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "\\10.16.10.220\Test Folder\" repository 562ab4bb opened successfully, password is correct error: CreateFile \?\UNC\10.16.10.220\Test Folder": The network name cannot be found. Fatal: unable to save snapshot: Lstat: CreateFile \?\UNC\10.16.10.220\Test Folder\10.16.10.220: The system cannot find the file specified.

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "\\10.16.10.220\Test Folder\*" repository 562ab4bb opened successfully, password is correct error: CreateFile \?\UNC\10.16.10.220\Test Folder*: The filename, directory name, or volume label syntax is incorrect.

Files: 0 new, 0 changed, 0 unmodified Dirs: 1 new, 0 changed, 0 unmodified Added to the repo: 328 B

processed 0 files, 0 B in 0:00 snapshot 0427e9f3 saved

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "\\10.16.10.220\TestFolderNoSpace" repository 562ab4bb opened successfully, password is correct Fatal: unable to save snapshot: Lstat: CreateFile \?\UNC\10.16.10.220\Test Folder\10.16.10.220: The system cannot find the file specified.

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "\\10.16.10.220\TestFolderNoSpace\" repository 562ab4bb opened successfully, password is correct

Files: 2 new, 0 changed, 0 unmodified Dirs: 0 new, 0 changed, 0 unmodified Added to the repo: 321 B

processed 2 files, 0 B in 0:00 snapshot 445b58ab saved

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "K:\TestFolderNoSpace" repository 562ab4bb opened successfully, password is correct

Files: 0 new, 0 changed, 2 unmodified Dirs: 0 new, 0 changed, 1 unmodified Added to the repo: 0 B

processed 2 files, 0 B in 0:00 snapshot 01349c2b saved

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "K:\Test Folder" repository 562ab4bb opened successfully, password is correct

Files: 2 new, 0 changed, 0 unmodified Dirs: 1 new, 0 changed, 0 unmodified Added to the repo: 563 B

processed 2 files, 0 B in 0:00 snapshot 56f82ecf saved

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test snapshots repository 562ab4bb opened successfully, password is correct

ID        Time                 Host        Tags        Paths
----------------------------------------------------------------------------------------
0427e9f3  2020-06-25 01:01:22  Core                    \\10.16.10.220\Test Folder\*
445b58ab  2020-06-25 01:01:39  Core                    \\10.16.10.220\TestFolderNoSpace\
01349c2b  2020-06-25 01:02:04  Core                    K:\TestFolderNoSpace
56f82ecf  2020-06-25 01:02:16  Core                    K:\Test Folder
----------------------------------------------------------------------------------------
4 snapshots

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test restore 0427e9f3 --target "K:\restore" repository 562ab4bb opened successfully, password is correct restoring <Snapshot 0427e9f3 of [\10.16.10.220\Test Folder*] at 2020-06-25 01:01:22.7598049 +0200 CEST by CORE\core@Core> to K:\restore ignoring error for : invalid child node name \10.16.10.220\Test Folder ignoring error for : invalid child node name \10.16.10.220\Test Folder There were 2 errors

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test restore 445b58ab --target "K:\restore" repository 562ab4bb opened successfully, password is correct restoring <Snapshot 445b58ab of [\10.16.10.220\TestFolderNoSpace] at 2020-06-25 01:01:39.8898124 +0200 CEST by CORE\core@Core> to K:\restore ignoring error for : invalid child node name \10.16.10.220\TestFolderNoSpace ignoring error for : invalid child node name \10.16.10.220\TestFolderNoSpace There were 2 errors

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test restore 01349c2b --target "K:\restore" repository 562ab4bb opened successfully, password is correct restoring <Snapshot 01349c2b of [K:\TestFolderNoSpace] at 2020-06-25 01:02:04.243429 +0200 CEST by CORE\core@Core> to K:\restore

> C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test restore 56f82ecf --target "K:\restore" repository 562ab4bb opened successfully, password is correct restoring <Snapshot 56f82ecf of [K:\Test Folder] at 2020-06-25 01:02:16.0841665 +0200 CEST by CORE\core@Core> to K:\restore

What backend/server/service did you use to store the repository?

Windows Path on Local Disk

Expected behavior

Restic cannot work with UNC Paths.

Actual behavior

1. There are minor problems at the backup Process when trying to backup a UNC Path with Spaces like: > C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "\\10.16.10.220\Test Folder" You'll have to use an additional backslash + wildcard like this: > C:\restic_0.9.6_w\restic.exe -r K:\ResticBackup\test backup "\\10.16.10.220\Test Folder\*" You don't need that with the Local Path equivalent: [...]"K:\Test Folder"

2. There is a major problem when you try to restore UNC Paths. It simply does not work and exit with:

ignoring error for \: invalid child node name \\10.16.10.220\TestFolderNoSpace
ignoring error for \: invalid child node name \\10.16.10.220\TestFolderNoSpace

Steps to reproduce the behavior

Do the exact same commands described above on a windows system with an elevated powershell.

Do you have any idea what may have caused this?

Restic cannot handle UNC Paths within powershell (also tested the restore with elevated cmd.exe)

Do you have an idea how to solve the issue?

No

Did restic help you today? Did it make you happy in any way?

You're doing an awesome job by developing this tool. And yes it did a very good job so far.

[fd0]

I'm very sorry for not having responded earlier. This is a bug in the backup code and the restorer trips over it. If you backup an UNC path directly, restic creates file names with the UNC path in it (which should not happen). The restorer has a mechanism to detect file names which contain path separators (such as \ on Windows) and refuses to restore them.

This patch should allow restoring, but the real bug in the backup code should be fixed:

diff --git internal/restorer/restorer.go internal/restorer/restorer.go
index 29224149..84cc36c7 100644
--- internal/restorer/restorer.go
+++ internal/restorer/restorer.go
@@ -62,14 +62,14 @@ func (res *Restorer) traverseTree(ctx context.Context, target, location string,
 		// ensure that the node name does not contain anything that refers to a
 		// top-level directory.
 		nodeName := filepath.Base(filepath.Join(string(filepath.Separator), node.Name))
-		if nodeName != node.Name {
-			debug.Log("node %q has invalid name %q", node.Name, nodeName)
-			err := res.Error(location, errors.Errorf("invalid child node name %s", node.Name))
-			if err != nil {
-				return hasRestored, err
-			}
-			continue
-		}
+		// if nodeName != node.Name {
+		// 	debug.Log("node %q has invalid name %q", node.Name, nodeName)
+		// 	err := res.Error(location, errors.Errorf("invalid child node name %s", node.Name))
+		// 	if err != nil {
+		// 		return hasRestored, err
+		// 	}
+		// 	continue
+		// }
 
 		nodeTarget := filepath.Join(target, nodeName)
 		nodeLocation := filepath.Join(location, nodeName

[AlBundy33]

@MichaelEischer / @fd0 wouldn't it be better to allow and remove or replace (e.g. with unc) \\ at the beginning of the path during restore (at least as hotfix)?

It's seems still not be fixed and users (including me) do backups of shared folders 🤔 https://forum.restic.net/t/invalid-child-node-name-in-windows-restore/5874

[MichaelEischer]

With restic 0.16.0 the subfolder syntax should be able to work around the problem: restore “latest:\10.16.10.220\Test Folder”

[jesvinjoachim]

i am not able to restore files to Windows, is this issue still unresolved? is the following the final solution or a temporary solution? With restic 0.16.0 the subfolder syntax should be able to work around the problem: restore “latest:\10.16.10.220\Test Folder”

[MichaelEischer]

is the following the final solution or a temporary solution?

It is only a workaround, hence a temporary solution. I've tried to solve the issue some time ago but that turned out to be more work than originally anticipated, so yes the issue still exists.

A proper fix will require modifying internal/archiver/tree.go to correctly handle volume names consisting of host and share name. restic automatically resolves collisions between backup target path names, which with UNC paths reaches a completely new level of complexity. For example what should happen when someone backs up restic backup host\share2 \\host\share1 \\host\share2 (note that the first host\share2 is not a UNC path). The result should likely be /host/share2, /host-1/share1 and /host-1/share2 (as paths within the snapshot). However, achieving that result will require a rewrite of the path collision handling.

[jesvinjoachim]

thank you, for the explanation, i understand, once again thank you and this will get the job done for me. I need a little help in using this

this is what I tried first Command execution failed because the underlying process (restic.exe#24468) returned a non-zero exit code (1).

Command: backup/restic ls 797f219c871dbe3d84fd39f108c3f7f37221870b1b211eee0376825ddd5de6e6 latest:\server\Share --cache-dir cache --json --quiet

You can suppress this validation by calling WithValidation(CommandResultValidation.None) on the command.

So instead

I should use the following right? backup/restic ls 797f219c871dbe3d84fd39f108c3f7f37221870b1b211eee0376825ddd5de6e6 latest:/server/Share --cache-dir cache --json --quiet

---

and when using ls

restic ls "797f219c871dbe3d84fd39f108c3f7f37221870b1b211eee0376825ddd5de6e6" //server/Share or restic ls "797f219c871dbe3d84fd39f108c3f7f37221870b1b211eee0376825ddd5de6e6" /server/Share or restic ls "797f219c871dbe3d84fd39f108c3f7f37221870b1b211eee0376825ddd5de6e6" \server\Share

I am very new to unc/linux/windows stuff .

[StefanBertels]

Just in case it helps (reducing conflicts for /host/share):

https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats#dos-device-paths

One option for alternative path in windows: \\?\UNC\Server\Share\Test\Foo.txt

I don't known restic path details. Would /?/UNC/host/share1 be allowed? Maybe it's possible to use those DOS device paths (or something similar) for UNC/local distinction.

[MichaelEischer]

backup/restic ls 797f219c871dbe3d84fd39f108c3f7f37221870b1b211eee0376825ddd5de6e6 latest:/server/Share --cache-dir cache --json --quiet

That command invocation cannot work. ls expects a snapshot id (optionally using the subfolder syntax) followed by a list of paths to filter for. That is restic ls latest, restic ls latest:/subfolder, restic ls 797f219c871dbe3d84fd39f108c3f7f37221870b1b211eee0376825ddd5de6e6 or restic ls latest /subfolder are valid syntax. In your case you'll likely want the following:

restic ls latest:\\server\Share/subfolder

(The UNC part uses backslashes, whereas subfolders within the volumes are separated by forward slashes)

I'm not entirely sure whether the first \\ should be a single or double backslash. You might have to try both variants.

The other ls syntax that might work is the following:

restic ls latest /\\server\Share/subfolder

The easiest way is probably to just run restic ls latest and use the paths exactly as printed there (you might have to escape backslashes though).

[jesvinjoachim]

Thank you very much, for the information you gave me, I worked it out and here are the results. it took a bit long because the s3 server had some issues and was very hard to test, just to confirm it was my syntax problem or s3 problem.

The following is 100% Working regarding ls

restic ls 7495c3e9be6b539533c912sdfds1b133e58e3102ac6e1651fa4977c5fbb83d0c05f /\\server\Share --cache-dir cache Now Regarding Restore, I followed a similar method, asls I need help in restoring too, something in my syntax is not right. Tried the methods followed below

A)

restic restore 7495c3e9be6b539533c912sdfds1b133e58e3102ac6e1651fa4977c5fbb83d0c05f --target C:\Users\Mypc\Desktop\test123 --include //server/Share/sample --cache-dir cache

B)

restic restore 7495c3e9be6b539533c912sdfds1b133e58e3102ac6e1651fa4977c5fbb83d0c05f --target C:\Users\Mypc\Desktop\test123 --include /\\server\Share/sample --cache-dir cache

C)

restic restore 7495c3e9be6b539533c912sdfds1b133e58e3102ac6e1651fa4977c5fbb83d0c05f --target C:\Users\Mypc\Desktop\test123 --include latest/\\server\Share/sample --cache-dir cache

D) restic restore 7495c3e9be6b539533c912521b133e58e3102ac6e1651fa4977c5fbb83d0c05f --target C:\Users\Mypc\Desktop\test123 --include latest:/\\server\Share/sample --cache-dir cache

E) restic restore latest --target C:\Users\Mypc\Desktop\test123 --include latest:/\\server\Share/sample --cache-dir cache

F) restic restore latest --target C:\Users\Mypc\Desktop\test123 --include latest:\server\Share/sample --cache-dir cache

G) restic restore latest --target C:\Users\Mypc\Desktop\test123 --include latest:\server\Share\sample --cache-dir cache

A, B,C,D F and G , are not working

Thank you.

[MichaelEischer]

A, B,C,D F and G , are not working

None of those variants use the snapshot:subfolder syntax in the correct place and therefore cannot work.

Let's take a closer look at restic restore latest --target C:\Users\Mypc\Desktop\test123 --include latest:\server\Share/sample --cache-dir cache:

• latest: that's the snapshot id, and also the place where the snapshot:subfolder syntax would have to be used.
• --include latest:\server\Share/sample: this tells restore to only include files from the folder latest:\server\Share/sample which cannot work. If server\Share shows up as part of the parameter to --include then it won't work.

What you want to use is something like the following: restic restore latest:/\\server\Share/ --target C:\Users\Mypc\Desktop\test123 --include /sample --cache-dir cache