vscode-remote-release

vscode-remote-release copied to clipboard

• VSCode Version: 1.67.2
• Local OS Version: Windows 10 21H2
• Remote OS Version: RHEL7.9
• Remote Extension/Connection Type: Containers
• Logs:

[5205 ms] TypeError: Cannot read properties of undefined (reading 'groups')
[5206 ms]     at ja (/home/ehripko/.vscode-remote-containers/dist/dev-containers-cli-0.238.2/dist/spec-node/devContainersSpecCLI.js:239:543)
[5206 ms]     at $u (/home/ehripko/.vscode-remote-containers/dist/dev-containers-cli-0.238.2/dist/spec-node/devContainersSpecCLI.js:188:3560)
[5206 ms]     at async RR (/home/ehripko/.vscode-remote-containers/dist/dev-containers-cli-0.238.2/dist/spec-node/devContainersSpecCLI.js:204:2322)
[5206 ms]     at async PR (/home/ehripko/.vscode-remote-containers/dist/dev-containers-cli-0.238.2/dist/spec-node/devContainersSpecCLI.js:188:2240)
[5206 ms]     at async zR (/home/ehripko/.vscode-remote-containers/dist/dev-containers-cli-0.238.2/dist/spec-node/devContainersSpecCLI.js:245:2177)
[5206 ms]     at async nb (/home/ehripko/.vscode-remote-containers/dist/dev-containers-cli-0.238.2/dist/spec-node/devContainersSpecCLI.js:245:3110)
[5206 ms]     at async JR (/home/ehripko/.vscode-remote-containers/dist/dev-containers-cli-0.238.2/dist/spec-node/devContainersSpecCLI.js:245:13046)
[5207 ms]     at async KR (/home/ehripko/.vscode-remote-containers/dist/dev-containers-cli-0.238.2/dist/spec-node/devContainersSpecCLI.js:245:12802)
[5209 ms] Exit code 1

VSC seems to assume that container files follow the Dockerfile syntax, but that is not necessarily the case as BuildKit supports custom frontends via #syntax=path/to/image:tag pragma. Some examples of custom frontends are:

• https://github.com/r2d4/mockerfile
• https://github.com/EricHripko/cnbp
• https://github.com/EricHripko/pack.yaml

Container files for these will start with the syntax= statement and follow a custom format afterwards. Based on my experiments, the issue appears to be triggered by updateRemoteUserUID feature.

Steps to Reproduce:

1. Make use of a custom BuildKit frontend in the container setup
2. Make sure that an unprivileged user is set as default and updateRemoteUserUID is on
3. Attempt to use VSC Remote - Containers
4. Observe the failure

Does this issue occur when you try this locally?: Yes Does this issue occur when you try this locally and all extensions are disabled?: Yes

Downgrading to v0.234 appears to solve the issue, so this seems to be a regression in the newer version of the extension. See below for a successful run with an older extension version:

[92225 ms] Generating composeOverrideFile...
[92228 ms] Start: Run: docker inspect --type image [redacted]_dev
[92350 ms] Start: Run: docker build -f /tmp/vsch/updateUID.Dockerfile-0.234.0 -t vsc-[redacted]-31145304a59342f6111c0fcadb79d702-uid --build-arg BASE_IMAGE=[redacted]_dev --build-arg REMOTE_USER=nobody --build-arg NEW_UID=1000 --build-arg NEW_GID=1000 --build-arg IMAGE_USER=root /tmp/vsch
[+] Building 0.9s (6/6) FINISHED                                                
 => [internal] load build definition from updateUID.Dockerfile-0.234.0     0.0s
 => => transferring dockerfile: 1.34kB                                     0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/library/[redacted]_dev:lat  0.0s
 => [1/2] FROM docker.io/library/[redacted]_dev                        0.3s
 => [2/2] RUN eval $(sed -n "s/nobody:[^:]*:\([^:]*\):\([^:]*\):[^:]*:\([  0.5s
 => exporting to image                                                     0.1s
 => => exporting layers                                                    0.1s
 => => writing image sha256:57e88c9f98d610e4bbb8a73817c5545ddc399f28cdbf1  0.0s
 => => naming to docker.io/library/vsc-[redacted]-31145304a59342f6111  0.0s
[93546 ms] Writing docker-compose.devcontainer.containerFeatures-1655827909087.yml to /tmp/docker-compose
[93548 ms] Start: Run: docker-compose --project-name [redacted] -f /home/ehripko/[redacted]/.dev-compose.yml -f /home/ehripko/[redacted]/.devcontainer/docker-compose.yml -f /tmp/docker-compose/docker-compose.devcontainer.containerFeatures-1655827909087.yml up -d

Hey folks! Is there any workaround we could use in the meantime? We're facing issues where some features/bugfixes in Remote - Containers require the latest version whilst this requires an old version. This makes using remote container workflows quite difficult as one needs to juggle between versions depending on what project you're working on and how.

CC @chrmarti as I can see you triaged this report.

This is also affecting some of our workflows as @EricHripko said. We need the newer versions to fix some other bugs, but then anything higher than 0.234 breaks with the above error. Is there anything more needed from users for triage ?

This is also an issue I'm facing. In my case I want to use the Dockerfile 1.4 syntax and despite my best efforts this doesn't seem possible right now.

Maybe this helps to figure out what is going wrong. The generated "Dockerfile-with-features" file does not contain the leading syntax marker. As can be seen given a Dockerfile input and the generate Dockerfile-with-features output.

# Some comment
# syntax=docker/dockerfile:1.4

Gets turned into

ARG _DEV_CONTAINERS_BASE_IMAGE=placeholder
# Some comment
# syntax=docker/dockerfile:1.4

But

# syntax=docker/dockerfile:1.4
# Some other comment
# syntax=docker/dockerfile:1.4

Turns into

ARG _DEV_CONTAINERS_BASE_IMAGE=placeholder
# Some other comment
# syntax=docker/dockerfile:1.4

Dev Containers 0.258.0-pre-release comes with a fix. Could you give that a try and let me know if it fixes it? Thanks.

The fix is to look at the syntax directive and only update it if it is before docker/dockerfile:1.4. If we don't recognize the syntax image, we also don't change it.

@chrmarti Not the OP but that fixed it for me—thank you!

Hi @chrmarti! Thank you for the update on this 👍 We tried it out and the issue seems to still be occurring on 0.258.0:

[1826 ms] TypeError: Cannot read properties of undefined (reading 'groups')
[1826 ms]     at Fc (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:216:14165)
[1826 ms]     at pd (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:241:3878)
[1826 ms]     at async rD (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:259:2425)
[1826 ms]     at async tD (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:241:2391)
[1826 ms]     at async ED (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:303:2193)
[1827 ms]     at async ys (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:303:3182)
[1827 ms]     at async zL (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:423:10319)
[1827 ms]     at async HL (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.258.0/dist/spec-node/devContainersSpecCLI.js:423:10075)
[1828 ms] Exit code 1

Is new version still assuming that there is FROM statement? This is not the case for the custom frontend we're using.

@EricHripko Indeed, is your frontend not compatible with the regular Dockerfile syntax?

@chrmarti, same firm as @EricHripko.

Our frontend is designed to not require a FROM statement. It decides the base image to use based on the build context. Since this is a supported use case for docker buildkit it would be awesome if this extension supported it as well.

100% Agree

@abid-mujtaba Could you append a simple example of such a Dockerfile? I'm trying to understand if we could still extend it in a way to add container features and metadata.

Current fix is now also available in Dev Containers 0.255.3.

@chrmarti here's a representative sample (with firm-specific stuff mocked out):

# syntax=foo.bar.com/an-org/some-frontend

COPY requirements-dev.txt requirements.txt /tmp/
RUN --mount=type=cache,target=/root/.cache \
        python3.10 -m pip install -r /tmp/requirements.txt -r /tmp/requirements-dev.txt

ENV PYTHONDONTWRITEBYTECODE 1

WORKDIR /workarea

This lives in a repo with Python requirements*.txt files and a pyproject.toml file so our frontend detects that it is a Python project and pulls in the appropriate base image automatically.

@abid-mujtaba Could you append a simple example of such a Dockerfile? I

Hi @chrmarti, same firm as @abid-mujtaba / @EricHripko .

I've used a one of the frontends that Eric has linked (https://github.com/EricHripko/pack.yaml) to create a minimal reproducible example. Steps to reproduce:

1. Checkout a GO repo, I've used https://github.com/gohugoio/hugo
2. Create a pack.yaml file with the following content:

# syntax = abdullahwali/pack.yaml
command: ["bash"]
user: "root"

3. Create .devcontainer/devcontainer.json

{
	"context": "..",
	"dockerFile": "../pack.yaml"
}

4. Attempt to reopen in a container:

this works on 0.234.0, but fails on later versions with:

[953 ms] TypeError: Cannot read properties of undefined (reading 'groups')
[953 ms]     at mm (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1787:14223)
[953 ms]     at Vse (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1868:2421)
[953 ms]     at async ZA (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1868:1973)
[954 ms]     at async jO (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1868:901)
[954 ms]     at async Xse (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1874:2030)
[954 ms]     at async Uf (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1874:3193)
[954 ms]     at async Cae (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1994:10350)
[954 ms]     at async yae (/Users/awali6/.vscode/extensions/ms-vscode-remote.remote-containers-0.255.3/dist/spec-node/devContainersSpecCLI.js:1994:10104)
[955 ms] Exit code 1

---

In our original use case, our custom frontend adds a base image on build time as @abid-mujtaba mentioned , which results in Dockerfiles without a FROM statement. However this example shows how with BuildKit + custom frontends, an image can be built without using the Dockerfile syntax at all, by using a configuration yaml file for example. This still results in a valid Docker image.

I'm trying to understand if we could still extend it in a way to add container features and metadata.

My suggestion would be to have an intermediate build step. Currently devcontainers seems to edit the Dockerfile to add the features and then build the resulting modified Dockerfile. This breaks when the original Dockerfile is using a custom syntax. An alternative solution would be to build the user specified Dockerfile as is first, tagging it as <tmp_image>, and then the extended Dockerfile with container features and metadata can start with

From <tmp_image>

# Remote containers business logic below
...etc

Hi @chrmarti, can we please reopen this issue as this is still occurring? I intend to open a PR to address this soon

Thanks @AbdullahWali for the details. We used to add container features in a separate build after building the user's Dockerfile and that had two disadvantages:

• Multi-platform builds would have to push the intermediate images to a registry (because the local image cache can only hold images for the local architecture).
• To share inline checksums (for caching with --cache-from), the intermediate image would also have to be pushed to the same registry as the final image (and listed in a --cache-from IIRC).

This makes me think that if we want to again support separate image builds, we will need an option in the devcontainer.json to enable it.

Supporting Dockerfiles without a leading FROM should be easier.

any headway on this issue? This currently prevents us from running Docker v2 (which will no longer allow opt out after April) since version 0.234.0 uses the old naming for compose containers (underscore vs dash)

Any update on this issue? Experiencing the same error on a similar set up. Downgrading to 0.234.0 solves the issue

There is a sort of workaround. This error only happens when vscode has to bring up the container(s) to begin with. If you bring the composable environment before you launch vscode it is quite happy connecting to the existing containers and this error doesn't show up.

Tested with VS Code 1.75.1 and Dev Containers extension v0.266.1.

Building on this one can in fact create a .devcontainers/initialize.sh shell script and then specify it as the initalizeCommand in .devcontainer/devcontainer.json. This script is run by the host, as the host, prior to vscode being injected into the container. An example of such a script is:

#! /usr/bin/env bash
#
# Initialization script for devcontainer
# This is run on the host by the host user NOT inside the container

init() {
    local -r SERVICE=<your_service_name>
    local -r CONTAINER_ID="$(docker-compose -f docker-compose.yaml -f .devcontainer/docker-compose.yaml ps -q ${SERVICE})"
    echo "Lab container id: ${CONTAINER_ID}"

    if [ -z "${CONTAINER_ID}" ]
    then
        echo "Bringing up container"
        docker-compose -f docker-compose.yaml -f .devcontainer/docker-compose.yaml up -d

        # Wait for container to come up
        wait_for_container "${SERVICE}"
    fi
}

wait_for_container() {
    local -r SERVICE="$1"

    echo "Waiting for container to come up"

    for _ in {1..30}
    do
        sleep 2

        if docker-compose -f docker-compose.yaml -f .devcontainer/docker-compose.yaml ps "${SERVICE}" | grep "Up"
        then
            echo "Container came up successfully. Proceeding."
            return 0
        fi
    done
}

init