docker compose compatibility - use current shell env for variable substitution in docker-compose.yaml

[johnklehm]

Description

docker compose will do env var substitutions based on the current shell env. To ensure compatibility we need to allow this behavior in nerdctl as well.

Steps to reproduce the issue

1. Have an env var set in your current shell env.
2. Reference this env var name in your docker-compose.yaml file
3. Run nerdctl up and you'll see the env var is not substituted

Describe the results you received and expected

The env var was marked as not found, we expect it to be present based on the existence in the env alone. e.g. a message like this is received indicating it's not there:

WARN[0000] The "POETRY_HTTP_BASIC_SHIPT_RESOLVE_PASSWORD" variable is not set. Defaulting to a blank string. 

but the env var is set in the current shell env.

The expectation is that the current shell env can be consulted for substitution candidates.

What version of nerdctl are you using?

Rancher desktop 1.4.1, nerdctl 0.20.0

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

Rancher Desktop for macOS

Host information

Client:
 Namespace:     default
 Debug Mode:    false

Server:
 Server Version: v1.5.11
 Storage Driver: overlayfs
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Log: json-file
  Storage: native overlayfs
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.15.40-0-virt
 Operating System: Alpine Linux v3.15
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 1.931GiB
 Name: lima-rancher-desktop
 ID: a5e68671-7f2d-473c-97f3-4de8042f672e

[ArVar]

I'm currently mitigating this by using --env-file .env. But the line env SOMETHING nerdctl compose build in my makefile indeed does not work well. So passing shell env variables would be nice for such cases as well.

[leocavalcante]

Same here. To reproduce is very simple:

Set an environment variable.

export FOO=bar
echo $FOO
bar

Try to build the image using the following Dockefile and docker-compose.yaml as examples.

FROM alpine
ARG FOO
ENV FOO $FOO

version: '3.9'
services:
  issue:
    container_name: nerdctl-issue
    image: nerdctl-issue
    build:
      context: .
      args:
        FOO: $FOO

Then

lima nerdctl compose build issue

Outputs

WARN[0000] The "FOO" variable is not set. Defaulting to a blank string.
WARN[0000] build.config should be relative path, got "/Users/leonardocavalcante/tmp/nerdctl-issue"
INFO[0000] Building image nerdctl-issue
[+] Building 1.7s (5/5) FINISHED
 => [internal] load .dockerignore                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                             0.0s
 => [internal] load build definition from Dockerfile                                                                                                        0.0s
 => => transferring dockerfile: 71B                                                                                                                         0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                                                                            1.4s
 => CACHED [1/1] FROM docker.io/library/alpine@sha256:69665d02cb32192e52e07644d76bc6f25abeb5410edc1c7a81a10ba3f0efb90a                                      0.1s
 => => resolve docker.io/library/alpine@sha256:69665d02cb32192e52e07644d76bc6f25abeb5410edc1c7a81a10ba3f0efb90a                                             0.1s
 => exporting to image                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                     0.0s
 => => exporting manifest sha256:d4b5804ec874a0d0691a84e1411baa250fd868444da1fa5d38858a1591e241b9                                                           0.0s
 => => exporting config sha256:8162ae4a226625f9a38c895362ffc05dde2e0f5567868231dbc62c2e014ba024                                                             0.0s
 => => naming to docker.io/library/nerdctl-issue:latest                                                                                                     0.0s
 => => unpacking to docker.io/library/nerdctl-issue:latest                                                                                                  0.0s

As you can see WARN[0000] The "FOO" variable is not set. Defaulting to a blank string. pops out already signaling that something is wrong.

Let's check it out anyway:

lima nerdctl run --rm -it nerdctl-issue
/ # echo $FOO

/ #

😢

---

But

Building without compose:

nerdctl build --build-arg=FOO=$FOO -t nerdctl-issue .

There are no warnings and:

➜  nerdctl-issue lima nerdctl run --rm -it nerdctl-issue
/ # echo $FOO
bar
/ #

Works!

[djdongjin]

@leocavalcante the issue is actually trickier.

nerdctl build --build-arg=FOO=$FOO -t nerdctl-issue . works because the $FOO in this command, is parsed by your shell, not by nerdctl/lima/etc. Simiarly to when you run echo $FOO, the $FOO is parsed by shell not by echo.

The issue with nerdctl (compose) is that the actual command is running not in your current shell so the env actually doesn't exist.

For example, I use nerdctl directly (to be accurate, sudo nerdctl in rootful mode). So export FOO=bar will be in my non-root environment, but sudo nerdctl compose build issue is running in root environment which doesn't share the same set of envs.

Similarly when you use lima I guess it might also be the case that nerdctl within lima vm, doesn't aware of your local envs at all.

To better elaborate using your example:

# explicitly add the env in root via `env`, so `Foo` is indeed in the root environment
# no the same warning
$ sudo env "FOO=bar" nerdctl compose build issue
WARN[0000] build.config should be relative path, got "/home/ec2-user/code/tmp/1152"
INFO[0000] Building image nerdctl-issue
[+] Building 0.2s (5/5) FINISHED
...
# the env is in the container
$ sudo nerdctl run -it --rm nerdctl-issue
/ # echo $FOO
bar

So the issue is how to make the current envs visible to the real environment nerdctl runs. For example,

for lima, how to make host envs visible in lima vm (not sure if it's done). for nerdctl, how to make current user envs visible in root when nerdctl is running in rootful mode (e.g. sudo nerdctl)

On a side note, docker doesn't have the same issue because docker did some extra setup for rootful (so you don't need sudo docker), but just for demo, if you run with sudo:

# the warning also appears.
$ sudo docker-compose build issue
WARNING: The FOO variable is not set. Defaulting to a blank string.
Building issue
Sending build context to Docker daemon  3.072kB
...

[leocavalcante]

Thank you very much, @djdongjin. Now I see it clearly. The nerdctl command runs inside the lima VM and the VM doesn't have the same environment variables as my host machine.

Do you know any workarounds? Should I just copy the env vars to the VM?

[djdongjin]

@leocavalcante I'm not sure about lima specifically, but this issue should be quite relevant: https://github.com/lima-vm/lima/issues/412

One workaround is lima seems support passing env variables in its config. So you can put your envs here:

https://github.com/lima-vm/lima/blob/cce35610a44033ac45d1833601c2b695f22c1b6d/examples/default.yaml#L365