`docker-in-docker` failing to fetch from https://packages.microsoft.com/repos/microsoft-debian-bookworm-prod

[georgewitteman]

Relevant logs:

[2023-06-15T18:47:23.429Z] #26 6.845 Processing triggers for libc-bin (2.36-9) ...

[2023-06-15T18:47:23.580Z] #26 6.893 update-alternatives: using /usr/sbin/iptables-legacy to provide /usr/sbin/iptables (iptables) in manual mode
#26 6.895 update-alternatives: using /usr/sbin/ip6tables-legacy to provide /usr/sbin/ip6tables (ip6tables) in manual mode
#26 6.899 MICROSOFT_GPG_KEYS_URI=https://packages.microsoft.com/keys/microsoft.asc

[2023-06-15T18:47:23.731Z] #26 7.039 Ign:1 https://packages.microsoft.com/repos/microsoft-debian-bookworm-prod bookworm InRelease
#26 7.041 Err:2 https://packages.microsoft.com/repos/microsoft-debian-bookworm-prod bookworm Release
#26 7.041   404  Not Found [IP: 40.74.238.15 443]
#26 7.051 Hit:3 http://deb.debian.org/debian bookworm InRelease
#26 7.053 Hit:4 http://deb.debian.org/debian bookworm-updates InRelease
#26 7.061 Hit:5 http://deb.debian.org/debian-security bookworm-security InRelease

[2023-06-15T18:47:23.848Z] #26 7.264 Reading package lists...
[2023-06-15T18:47:24.601Z] 
#26 7.959 E: The repository 'https://packages.microsoft.com/repos/microsoft-debian-bookworm-prod bookworm Release' does not have a Release file.
#26 7.960 ERROR: Feature "Docker (Docker-in-Docker)" (ghcr.io/devcontainers/features/docker-in-docker) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/docker-in-docker for help troubleshooting this error.

[2023-06-15T18:47:24.624Z] #26 ERROR: executor failed running [/bin/sh -c cp -ar /tmp/build-features-src/docker-in-docker_2 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/docker-in-docker_2  && cd /tmp/dev-container-features/docker-in-docker_2  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/docker-in-docker_2]: exit code: 100

#12 [dev_container_auto_added_stage_label 1/7] FROM mcr.microsoft.com/devcontainers/javascript-node:18@sha256:c9ae77db2bc23935e47c73cd1f346c8bbf235ac1c108ed1f676beedf951a50e1
------
 > [dev_containers_target_stage 6/9] RUN --mount=type=bind,from=dev_containers_feature_content_source,source=docker-in-docker_2,target=/tmp/build-features-src/docker-in-docker_2     cp -ar /tmp/build-features-src/docker-in-docker_2 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/docker-in-docker_2  && cd /tmp/dev-container-features/docker-in-docker_2  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/docker-in-docker_2:
#26 6.899 MICROSOFT_GPG_KEYS_URI=https://packages.microsoft.com/keys/microsoft.asc
#26 7.039 Ign:1 https://packages.microsoft.com/repos/microsoft-debian-bookworm-prod bookworm InRelease
#26 7.041 Err:2 https://packages.microsoft.com/repos/microsoft-debian-bookworm-prod bookworm Release
#26 7.041   404  Not Found [IP: 40.74.238.15 443]
#26 7.051 Hit:3 http://deb.debian.org/debian bookworm InRelease
#26 7.053 Hit:4 http://deb.debian.org/debian bookworm-updates InRelease
#26 7.061 Hit:5 http://deb.debian.org/debian-security bookworm-security InRelease

#26 7.959 E: The repository 'https://packages.microsoft.com/repos/microsoft-debian-bookworm-prod bookworm Release' does not have a Release file.
#26 7.960 ERROR: Feature "Docker (Docker-in-Docker)" (ghcr.io/devcontainers/features/docker-in-docker) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/docker-in-docker for help troubleshooting this error.
------
ERROR: failed to solve: executor failed running [/bin/sh -c cp -ar /tmp/build-features-src/docker-in-docker_2 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/docker-in-docker_2  && cd /tmp/dev-container-features/docker-in-docker_2  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/docker-in-docker_2]: exit code: 100

[georgewitteman]

Looks like setting "moby": false is a workaround

[joshspicer]

Thanks for raising! I've also run into this and looking into a fix.

Edit: Contacted the upstream team to coordinate releasing the required Debian bookworm packages.

---

Note that this is because several of the images we maintain (from devcontainers/images) recently had a major revision to them that updates the base OS to Debian Bookworm. You can also get around this temporarily for many of our images by pinning your images to a non-bookworm image.

For example, for typescript-node, I confirmed that using a tag that pins to bullseye (the previous version of debian) works:

{
  "image": "mcr.microsoft.com/devcontainers/typescript-node:1-16-bullseye",
  "features": {
     "ghcr.io/devcontainers/features/docker-in-docker:2": {}
  }
}

All of the tags can be found in the associated image readmes: https://mcr.microsoft.com/v2/devcontainers/typescript-node/tags/list

In general, we always recommend pinning to the most specific image tag feasible for your project

[joshspicer]

@georgewitteman - What base image are you using? You should be able to take my suggestion above to pin to a more specific image.

[joshspicer]

The Feature installs moby from the Microsoft feeds, which aren't updated for debian bookworm yet

[joshspicer]

Opened notice https://github.com/devcontainers/features/issues/576 to catch related feedback

[georgewitteman]

@georgewitteman - What base image are you using? You should be able to take my suggestion above to pin to a more specific image.

That's it! We're using mcr.microsoft.com/devcontainers/javascript-node:18 (unpinned). Thanks for the help!

[metaskills]

FWIW, I was using the node:18-bookworm image and I worked around whatever my issue was by doing this in my devcontainer/Dockerfile:

RUN apt-get install -y pipx && \
    rm /usr/lib/python3.11/EXTERNALLY-MANAGED

[metaskills]

FYI, updated to these two latest features and was able to remove the above work around nor seeing the orig issue.

"ghcr.io/devcontainers/features/common-utils:2.4.2": {},
"ghcr.io/devcontainers/features/docker-in-docker:2.10.0": {},