Snapshotter does not create expected directory prior to extracting layer

[glerb]

Description

When pulling from a repository, layers download successfully, but extraction of layers fails with The system cannot find the path specified. The directory C:\ProgramData\containerd\root\io.containerd.snapshotter.v1.windows\snapshots\1 is expected, but the final leaf .\1 does not exist. Manually creating this subdirectory results in the same error on a subsequent pull, only a leaf of .\2 is expected and not present.

Steps to reproduce the issue

1. nerdctl.exe pull <repo>:<tag>

Describe the results you received and expected

Received:

time="2024-01-11T20:58:24-08:00" level=fatal msg="failed to extract layer sha256:<digest>: hcsshim::ProcessBaseLayer \\\\?\\C:\\ProgramData\\containerd\\root\\io.containerd.snapshotter.v1.windows\\snapshots\\1: The system cannot find the path specified.: unknown"

Expected:

Successful download and extraction of image layers.

What version of nerdctl are you using?

C:\..\nerdctl.exe version
time="2024-01-12T00:15:47-08:00" level=warning msg="unable to determine buildctl version: exec: \"buildctl\": executable file not found in %PATH%"
time="2024-01-12T00:15:47-08:00" level=warning msg="unable to determine runc version: exec: \"runc\": executable file not found in %PATH%"
Client:
 Version:       v1.7.2
 OS/Arch:       windows/amd64
 Git commit:    e32c4b023bf41e5c8325cfb893a53cefb5fc68ed
 buildctl:
  Version:

Server:
 containerd:
  Version:      v1.6.4
  GitCommit:    212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
 runc:
  Version:

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

c:\..\nerdctl.exe info
Client:
 Namespace:     default
 Debug Mode:    false

Server:
 Server Version: v1.6.4
 Storage Driver: windows
 Logging Driver: json-file
 Cgroup Driver:
 Cgroup Version:
 Plugins:
  Log: fluentd journald json-file syslog
  Storage: windows-lcow windows
 Security Options:
 Kernel Version:
 Operating System:
 OSType: windows
 Architecture:
 CPUs: 0
 Total Memory: 0B

[glerb]

Might be a containerd issue: https://github.com/containerd/containerd/blob/0a7c192ca95df3dac19b2b18d0f1f94083af9e14/pkg/unpack/unpacker.go#L384

[iankingori]

@glerb which image in particular were you trying to pull?

[phhpe]

This is an issue for any image.

[TinaMor]

@glerb (cc. @phhpe)

Thank you for bringing the issue to our attention.

I've been unable to replicate the problem with both version 1.7.2 and the latest version v1.7.6. The issue appears to be fixed in the latest containerd version v1.7.17

To ensure that we address this issue, could you please confirm whether you are still experiencing the problem on your end?

Using nerdctl version version 1.7.2 and containerd version v1.6.4

Using version version 1.7.2 and the latest containerd version v1.7.17

[TBBle]

Containerd 1.6.4 definitely should have been creating the snapshot directory automatically, so I'm not sure what could possibly have been going on here.

One possibility is that a virus scanner was blocking access to the new directory pending some scan activity, although that sort of this was more-commonly seen when trying to access just-extracted files, rather than new, empty directories. (That would explain why the final part of the error is "unknown" rather than "No such file or directory".)

Either way, this is definitely a containerd responsibility, not nerdctl, and barring a current-version reproduction case, we can probably close this report.