Error running non-stargz image

[aledbf]

trafficstars

I am trying to run an image and I get an error

Feb 23 12:25:17 g1d4432f5d44ad3f0602c3b containerd-stargz-grpc[37819]: {"error":"failed to resolve layer: failed to resolve layer \"sha256:e9444ec2d0d74706e9f21c69197f72e2b34056bf07c22e6e0590eeee3746eb75\" from \"reg.g1d4432f5d44ad3f0602c3b.workspace-preview.io:30000/remote/f1e4fe39-fa4d-4ed0-b0d2-023e9ded94c0:latest\": failed to resolve the blob: failed to resolve the source: cannot resolve layer: failed to get size (host \"reg.g1d4432f5d44ad3f0602c3b.workspace-preview.io:30000\", ref:\"reg.g1d4432f5d44ad3f0602c3b.workspace-preview.io:30000/remote/f1e4fe39-fa4d-4ed0-b0d2-023e9ded94c0:latest\", digest:\"sha256:e9444ec2d0d74706e9f21c69197f72e2b34056bf07c22e6e0590eeee3746eb75\"): strconv.ParseInt: parsing \"\": invalid syntax: failed to resolve: failed to resolve target","key":"k8s.io/73/extract-488070563-hqVv sha256:6d16c2897dc84bc73fa5b4cbf6a69056cfc851dda6278178d58001cdb9394311","level":"warning","msg":"failed to prepare remote snapshot","parent":"k8s.io/72/sha256:36ffdceb4c77bf34325fb695e64ea447f688797f2f1e3af224c29593310578d2","remote-snapshot-prepared":"false","time":"2022-02-23T12:25:17.018600807Z"}

any idea how to track the source?

The image works just fine without stargz-snapshotter Thanks!

[ktock]

@aledbf Thank you for reporting the issue.

The error seems to come from here. During mounting a layer, stargz snapshotter tries to get size of that layer by sending HEAD to /v2/<name>/blobs/<digest> endpoint of the registry. And stargz snapshotter expects the 200 response to contain Content-Length header (e.g. docker registry supports this header). However, stargz snapshotter seems to fail to get that header from your registry.

Does your registry response Content-Length to the HEAD request to /v2/<name>/blobs/<digest> endpoint (maybe reg.g1d4432f5d44ad3f0602c3b.workspace-preview.io:30000/v2/remote/f1e4fe39-fa4d-4ed0-b0d2-023e9ded94c0/blobs/sha256:e9444ec2d0d74706e9f21c69197f72e2b34056bf07c22e6e0590eeee3746eb75 according to the log shown above)?

[ktock]

I noticed that your log shows "remote-snapshot-prepared":"false" log and this is an expected behaviour for non-stargz images. I believe stargz snapshotter should fallback to (non-lazy) overlayfs-mode. Are there any error logs on containerd side?