Cannot find 'blobs/sha256/...' in parsed layers using dive 0.12.0

[zevisert]

What happened:

I'm still seeing this OCI layout error with dive that #500 was supposed to fix.

$ dive --source docker-archive image.tar         
Image Source: docker-archive://image.tar
Fetching image... (this can take a while for large images)
cannot fetch image
could not find 'blobs/sha256/4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1' in parsed layers

What you expected to happen:

dive should be able to handle these OCI layout images in v0.12.0+

How to reproduce it (as minimally and precisely as possible):

Here is the source dockerfile, but I've attached a prebuilt image in case.

FROM alpine:3.11.6 as base-image
ARG TARGETARCH

RUN --mount=type=cache,id=apk-${TARGETARCH},sharing=locked,target=/var/cache/apk \
    ln -s /var/cache/apk /etc/apk/cache && \
    ls -lah /var/cache/apk && \
    apk --update add bash

RUN uname -a

1. download this pre-built image: image.tar.gz
2. unzip the downloaded image: gunzip image.tar.gz
3. run dive --source docker-archive image.tar
4. Optionally: extract the image to look at layers tar xf image.tar -C image
5. Optionally: load the image into docker and run dive on the loaded image (amd64 only) docker load < image.tar
   • dive tar.dive.example.com/image:latest

Anything else we need to know?:

Environment:

• Dive version: 0.12.0
• OS version: NixOS 23.11 (Tapir)
• Docker version (if applicable):

Client:
 Cloud integration: v1.0.35+desktop.10
 Version:           25.0.3
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        4debf41
 Built:             Tue Feb  6 21:13:00 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Desktop
 Engine:
  Version:          25.0.3
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       f417435e5f6216828dec57958c490c4f8bae4f98
  Built:            Wed Feb  7 00:39:16 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[zevisert]

Ultimately, instead of guessing at layer encodings like https://github.com/wagoodman/dive/blob/925cdd86482edec42185794620a1e616b79bbee5/dive/image/docker/image_archive.go#L52 is doing, why not parse OCI images as they're meant to be?

If we added a quick pass over the archive files to first check if it contains an oci-layout file, if it does read index.json next and follow the manifests from there. Each digest points into the blobs subfolder, and each digest comes with a mediaType that tells us how to decode that blob (eg JSON, TAR, TAR+GZIP, etc).

[tbroyer]

If we added a quick pass over the archive files to first check if it contains an oci-layout file, if it does read index.json next and follow the manifests from there.

Unless you give it a TAR file, the TAR is directly streamed from the Docker daemon, so doing a 2-pass parsing would mean storing it in a temporary location (and risking filling up the disk/partition)

[zevisert]

Thanks @tbroyer, good point. FWIW #511 does fix this issue for me - with those changes as a patch to my personal build of dive everything seems to be working normally now

[erickguan]

I encountered a similar issue. I use docker containerd image storage and zstd OCI image. When I build image locally on macOS, dive reports:

Image Source: docker://test:debug
Fetching image... (this can take a while for large images)
cannot fetch image
could not find 'blobs/sha256/022ae4fed1b225b067d78e005f79c63bc1f5281747d428a534c5cf2c4d025028' in parsed layers

When pulling a docker image from the registry, dive reports:

dive ghcr.io/skvk/nexus
Image Source: docker://test
Fetching image... (this can take a while for large images)
cannot fetch image
could not find image config

docker info:

Client:
 Version:    25.0.3
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1-desktop.4
    Path:     /Users/erickg/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.6-desktop.1
    Path:     /Users/erickg/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container. (Docker Inc.)
    Version:  0.0.24
    Path:     /Users/erickg/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/erickg/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.22
    Path:     /Users/erickg/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/erickg/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.0.1
    Path:     /Users/erickg/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/erickg/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.5.0
    Path:     /Users/erickg/.docker/cli-plugins/docker-scout

Server:
 Containers: 33
  Running: 1
  Paused: 0
  Stopped: 32
 Images: 30
 Server Version: 25.0.3
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.16-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 10.69GiB
 Name: docker-desktop
 ID: 575b1def-c572-44cb-813c-4a614c4adf31
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

[ohmer]

Same issue with latest Docker Desktop on MacOS:

 Cloud integration: v1.0.35+desktop.13
 Version:           26.0.0
 API version:       1.45
 Go version:        go1.21.8
 Git commit:        2ae903e
 Built:             Wed Mar 20 15:14:46 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.29.0 (145265)
 Engine:
  Version:          26.0.0
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.8
  Git commit:       8b79278
  Built:            Wed Mar 20 15:18:02 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

https://github.com/wagoodman/dive/pull/511 fixed it for me as well.