podman icon indicating copy to clipboard operation
podman copied to clipboard

podman build --secret help doesn't match docs and does not work for type=env

Open apollo13 opened this issue 1 year ago • 9 comments

Issue Description

I am trying to pass some secret into a container build without leaving any traces in the container itself. Playing around I tried:

podman build --secret secret,type=env,target=ASD -f Containerfile.test .

which complains:

Error: creating build executor: incorrect secret flag format: should be --secret id=foo,src=bar[,env=ENV,type=file|env]

So far so good, prefixing with id= successfully builds but doesn't expose the env to the container. The docs seem to suggest that this won't work anyways https://docs.podman.io/en/latest/markdown/podman-build.1.html#secret-id-id-src-path and apparently only supported for podman run/create: https://docs.podman.io/en/latest/markdown/podman-run.1.html#secret-secret-opt-opt

So I am assuming that podman build uses the same "parsing logic" while not really supporting all of it.

Ideally I'd like to be able to set a transient env variable during a container build.

Steps to reproduce the issue

$ cat Containerfile.test
FROM python:3.12-slim
RUN echo $ASD
$ podman build --no-cache --secret id=secret,type=env,target=ASD -f Containerfile.test . 
STEP 1/2: FROM python:3.12-slim
STEP 2/2: RUN echo $ASD

COMMIT
--> 2dd962b49c90
2dd962b49c90baa044a1dcb1887baf9c546622dcfa5f4e73c1270df8236630fb

Describe the results you received

$ASD is empty inside the build context

Describe the results you expected

I'd love to see the content of the secret

podman info output

host:
  arch: amd64
  buildahVersion: 1.36.0
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.fc40.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 97.82
    systemPercent: 0.67
    userPercent: 1.51
  cpus: 16
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: workstation
    version: "40"
  eventLogger: journald
  freeLocks: 2045
  hostname: apollo13
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 6.9.9-200.fc40.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 4261511168
  memTotal: 32399306752
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.11.0-1.fc40.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.11.0
    package: netavark-1.11.0-1.fc40.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.11.0
  ociRuntime:
    name: crun
    package: crun-1.15-1.fc40.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.15
      commit: e6eacaf4034e84185fd8780ac9262bbf57082278
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240624.g1ee2eca-1.fc40.x86_64
    version: |
      pasta 0^20240624.g1ee2eca-1.fc40.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: false
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 7h 55m 7.00s (Approximately 0.29 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /home/florian/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/florian/.local/share/containers/storage
  graphRootAllocated: 536854134784
  graphRootUsed: 288820391936
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 417
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/florian/.local/share/containers/storage/volumes
version:
  APIVersion: 5.1.1
  Built: 1717459200
  BuiltTime: Tue Jun  4 02:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.3
  Os: linux
  OsArch: linux/amd64
  Version: 5.1.1


### Podman in a container

No

### Privileged Or Rootless

None

### Upstream Latest Release

No

### Additional environment details

_No response_

### Additional information

_No response_

apollo13 avatar Jul 24 '24 14:07 apollo13

So you would expect that each RUN line would have access to the secret environment variable.

rhatdan avatar Jul 24 '24 14:07 rhatdan

@flouthoc @nalind WDYT?

rhatdan avatar Jul 24 '24 14:07 rhatdan

@rhatdan Yes, that would be nice. To elaborate: I am trying to set environment variables for common package managers (like python's pip and npm) so that the container builds use our internal mirrors without having to manually add support for that inside the Containerfile (currently I have a build-arg ala: ARG PIP_PYPI_URL=https://pypi.org/simple but that leaks into the container history if I override it from the outside -- unless I use it in an intermediate container in multi stage builds.).

I fully understand though if that stays only supported for podman run/create. Nevertheless it would be really nice to have and might somewhat improve security as well.

apollo13 avatar Jul 24 '24 14:07 apollo13

Yes I think this is reasonable.

rhatdan avatar Jul 24 '24 14:07 rhatdan

@ashley-cui WDYT?

rhatdan avatar Jul 24 '24 14:07 rhatdan

@apollo13 I think --mount=type=secret only exposes secret for a specific RUN and not creates secret for entire container, it will expose it on the dst path but will be transient in nature and you can manually export it to your preferred env variable.

Please feel free to close the issue if you think above should suffice your use-case.

flouthoc avatar Aug 13 '24 15:08 flouthoc

@flouthoc Yes, that works, but requires cooperation from inside the docker file. With my proposed solution I could "enforce" a company wide pypi mirror (etc) without manually touching all the dockerfiles. Also the fallback is rather ugly if you mount something in RUN it probably has to exist in some form.

apollo13 avatar Aug 14 '24 08:08 apollo13

I think setting the environment variable for the length of the RUN command would be a good idea.

rhatdan avatar Aug 16 '24 22:08 rhatdan

A friendly reminder that this issue had no activity for 30 days.

github-actions[bot] avatar Sep 18 '24 00:09 github-actions[bot]

Ideally I'd like to be able to set a transient env variable during a container build.

+1 I've run into a similar use case with podman build in the context of a CI engine which is already making secrets available as environment variables. To get them into podman, they first have to be written to files to be subsequently imported as secrets.

ralphie79 avatar Nov 08 '24 22:11 ralphie79

Ideally I'd like to be able to set a transient env variable during a container build.

+1 I've run into a similar use case with podman build in the context of a CI engine which is already making secrets available as environment variables. To get them into podman, they first have to be written to files to be subsequently imported as secrets.

@ralphie79 - It is possible to do a podman build using secrets as environment variables. I was also struggling to figure this out and once I did, I submitted a PR to improve the docs around secrets, including examples. The example you are looking for can be seen HERE

hdub-tech avatar Jan 24 '25 16:01 hdub-tech

Hi all!

Can someone please ELI5 how I can get a secret that my CI server exposes as env var into my podman build?

To understand it, I literally copy-pasted the updated example from the docs and it doesn't work.

export MYSECRET=foobar
podman build --secret=id=mysecret,src=MYSECRET,type=env .

Error: open MYSECRET: no such file or directory

so I tried the shorter version. This time podman at least starts the build, but the secret is empty.

export MYSECRET=foobar
podman build --secret=id=mysecret,env=MYSECRET
....
STEP 6/6: RUN --mount=type=secret,id=mysecret     ls -la /run/secrets &&     cat /run/secrets/mysecret
total 0
drwxr-xr-x    3 root     root            34 Feb  7 10:55 .
drwxr-xr-x    1 root     root            42 Feb  7 10:55 ..
-r--------    1 root     root             0 Feb  7 10:55 mysecret
drwxr-xr-x    4 root     root            51 Feb  7 10:55 rhsm
COMMIT
--> a2c99b0fa91b

mpern avatar Feb 07 '25 10:02 mpern

@mpern - is your Containerfile using a non-privileged user at that step? I would have included this in the PR, but I discovered it afterwards: If it is an unprivileged user, you need to edit the RUN --mount to specify the uid of the user which you want to be able to read the secret:

RUN --mount=type=secret,id=mysecret,uid=$UNPRIVILEGED_UID 

Edit to add: I thought I had figured that out from a buildah documentation, but I can only find proof of that (and the other available tags) in the source:

https://github.com/containers/buildah/blob/3541a3ce2a5bacc0643ed87d501bf3c9bd5256ca/run_common.go#L1766 (commit = main as of this writing)

secret should have syntax id=id[,target=path,required=bool,mode=uint,uid=uint,gid=uint

hdub-tech avatar Feb 07 '25 17:02 hdub-tech

@hdub-tech - no non-priviliged user involved, I'm just trying to figure out how to get the env var as secret into the build in the first place.

Possibly related? containers/buildah#5808

FROM alpine
RUN --mount=type=secret,id=mysecret \
    ls -la /run/secrets && \
    cat /run/secrets/mysecret

Attempt 1 - type=env

export MYSECRET=foobar && podman build --no-cache --secret=id=mysecret,src=MYSECRET,type=env .
Error: open MYSECRET: no such file or directory

so, type=env doesn't work at all :(

Attempt 2 - env=...

export MYSECRET=foobar && podman build --no-cache --secret=id=mysecret,env=MYSECRET .
...
STEP 2/2: RUN --mount=type=secret,id=mysecret     ls -la /run/secrets &&     cat /run/secrets/mysecret
total 0
drwxr-xr-x    3 root     root            34 Feb 10 10:33 .
drwxr-xr-x    1 root     root            42 Feb 10 10:33 ..
-r--------    1 root     root             0 Feb 10 10:33 mysecret
drwxr-xr-x    4 root     root            51 Feb 10 10:33 rhsm
COMMIT

check the output of ls - mysecret has zero bytes.

Conclusion

Setting build secrets via environment variables simply doesn't work. 😭

Workaround:

echo to (temp) file and use file-based approach

export MYSECRET=foobar && echo "$MYSECRET" > mysecret && podman build --no-cache --secret=id=mysecret,src=mysecret .
...
STEP 2/2: RUN --mount=type=secret,id=mysecret     ls -la /run/secrets &&     cat /run/secrets/mysecret
total 4
drwxr-xr-x    3 root     root            34 Feb 10 10:37 .
drwxr-xr-x    1 root     root            42 Feb 10 10:37 ..
-r--------    1 root     root             7 Feb 10 10:37 mysecret
drwxr-xr-x    4 root     root            51 Feb 10 10:37 rhsm
foobar
COMMIT

podman info

podman info output
host:
  arch: arm64
  buildahVersion: 1.38.1
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-3.fc41.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 99.83
    systemPercent: 0.09
    userPercent: 0.07
  cpus: 6
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "41"
  eventLogger: journald
  freeLocks: 2048
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 6.12.7-200.fc41.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 16262627328
  memTotal: 16722104320
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.13.1-1.fc41.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.13.1
    package: netavark-1.13.1-1.fc41.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.13.1
  ociRuntime:
    name: crun
    package: crun-1.19.1-1.fc41.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.19.1
      commit: 3e32a70c93f5aa5fea69b50256cca7fd4aa23c80
      rundir: /run/user/501/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20241211.g09478d5-1.fc41.aarch64
    version: |
      pasta 0^20241211.g09478d5-1.fc41.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: unix:///run/user/501/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.3.1-1.fc41.aarch64
    version: |-
      slirp4netns version 1.3.1
      commit: e5e368c4f5db6ae75c2fce786e31eef9da6bf236
      libslirp: 4.8.0
      SLIRP_CONFIG_VERSION_MAX: 5
      libseccomp: 2.5.5
  swapFree: 0
  swapTotal: 0
  uptime: 0h 17m 48.00s
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphRootAllocated: 106415992832
  graphRootUsed: 5352988672
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 75
  runRoot: /run/user/501/containers
  transientStore: false
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 5.3.2
  Built: 1737504000
  BuiltTime: Wed Jan 22 01:00:00 2025
  GitCommit: ""
  GoVersion: go1.23.4
  Os: linux
  OsArch: linux/arm64
  Version: 5.3.2

mpern avatar Feb 10 '25 10:02 mpern

@mpern - apologies on the delayed response, I'm hoping you figured this out already, but in case you did not, here are some of my findings. Also DISCLAIMER: I Am Not A Podman Expert by any means: this was just a bit of dive I took on something that was driving me crazy and I managed to solve for my own project, and attempted to share back :sweat_smile:

TL;DR: All of your attempts "worked on my box" in Debian 12 with a Debian 12 Container (which had been my project env) and in Fedora 41 cloud with an Alpine container (to attempt to mirror yours).

~Edit to add: I don't think https://github.com/containers/buildah/issues/5808 is an actual issue, and I commented on it to hopefully help the OP of it.~


Environment

For the sake of completeness, here is the vagrant VM I was using:

Vagrantfile:

$osname = "fedora"
$vmname = "#{$osname}-41-cloud-base"
Vagrant.configure("2") do |config|

  config.vm.define "#{$vmanme}" do |f|
    f.vm.box = "fedora/41-cloud-base"
    f.vm.hostname = $osname

    # VirtualBox configuration
    f.vm.provider "virtualbox" do |v|
      # UI name in VirtualBox
      v.name = $vmname
      # Hungry hungry hippo
      v.memory = 4096
    end

    f.vm.provision "Install podman", type: "shell", inline: <<~DNF_SCRIPT
      dnf --assumeyes upgrade
      dnf --assumeyes install podman
    DNF_SCRIPT
  end
end

And I copied your Containerfile exactly

FROM alpine
RUN --mount=type=secret,id=mysecret \
    ls -la /run/secrets && \
    cat /run/secrets/mysecret

My podman info reflects yours fairly closely, with the big exceptions being (Again, not an expert and not sure which specifically, if any, are relevant and I wrote these down just because I am SUPER interested in why mine works and yours does not, and maybe this summary will help someone else put you on the correct path):

  • I am on amd64 and you are on arm
  • I used a cloud build of fedora
  • You had slirp4netns and I did not
  • My backing filesystem was btrfs and yours was zfs (I noticed an issue that indicates zfs might have issues?)
  • Same podman, buildah, go, conmon, crun, netavark versions.
  • Slightly different kernel and pasta versions.
host:
  arch: amd64
  buildahVersion: 1.38.1
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-3.fc41.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 98.69
    systemPercent: 0.9
    userPercent: 0.41
  cpus: 1
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: cloud
    version: "41"
  eventLogger: journald
  freeLocks: 2048
  hostname: fedora
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 6.12.11-200.fc41.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 3669921792
  memTotal: 4095356928
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.13.1-1.fc41.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.13.1
    package: netavark-1.13.1-1.fc41.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.13.1
  ociRuntime:
    name: crun
    package: crun-1.19.1-1.fc41.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.19.1
      commit: 3e32a70c93f5aa5fea69b50256cca7fd4aa23c80
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20250121.g4f2c8e7-2.fc41.x86_64
    version: |
      pasta 0^20250121.g4f2c8e7-2.fc41.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 4094685184
  swapTotal: 4094685184
  uptime: 0h 16m 56.00s
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /home/vagrant/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/vagrant/.local/share/containers/storage
  graphRootAllocated: 4212109312
  graphRootUsed: 1015201792
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 3
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/vagrant/.local/share/containers/storage/volumes
version:
  APIVersion: 5.3.2
  Built: 1737504000
  BuiltTime: Wed Jan 22 00:00:00 2025
  GitCommit: ""
  GoVersion: go1.23.4
  Os: linux
  OsArch: linux/amd64
  Version: 5.3.2

Attempts

(NOTE: I'm skipping sharing the attempts from my Debian tests since they had the same results, and are not reflective of the issues you are facing)

Attempt 1 - type=env

[vagrant@fedora ~]$ export MYSECRET=foobar && podman build --no-cache --secret=id=mysecret,src=MYSECRET,type=env .
STEP 1/2: FROM alpine
STEP 2/2: RUN --mount=type=secret,id=mysecret     ls -la /run/secrets &&     cat /run/secrets/mysecret
total 4
drwxr-xr-x    1 root     root            16 Feb 21 23:51 .
drwxr-xr-x    1 root     root            40 Feb 21 23:51 ..
-r--------    1 root     root             6 Feb 21 23:51 mysecret
foobarCOMMIT
--> c76e30c00055
c76e30c0005585e887de2279d8d937bc095edaafc5b0f79f678615457838f898

Attempt 2 - env=...

[vagrant@fedora ~]$ export MYSECRET=foobar && podman build --no-cache --secret=id=mysecret,env=MYSECRET .
STEP 1/2: FROM alpine
STEP 2/2: RUN --mount=type=secret,id=mysecret     ls -la /run/secrets &&     cat /run/secrets/mysecret
total 4
drwxr-xr-x    1 root     root            16 Feb 21 23:54 .
drwxr-xr-x    1 root     root            40 Feb 21 23:54 ..
-r--------    1 root     root             6 Feb 21 23:54 mysecret
foobarCOMMIT
--> 369dd62fc8b0
369dd62fc8b0cb51a90cbe53b62e0e7de01541d8453a9982a2d89595ae6a940f

hdub-tech avatar Feb 22 '25 00:02 hdub-tech

I know this is a thread on Linux, I just wanted to mention that this is also not working on Windows with WSL2. The mounted secret file is 0 bytes:

>echo %MY_SECRET%
1234

>podman build --secret=id=mysecret,env=MY_SECRET -f Containerfile.test . --no-cache
STEP 1/2: FROM python:3.12-slim
STEP 2/2: RUN --mount=type=secret,id=mysecret  ls -la /run/secrets && cat /run/secrets/mysecret
total 12
drwxr-xr-x 3 root root 4096 Feb 24 15:55 .
drwxr-xr-x 1 root root 4096 Feb 24 15:55 ..
-r-------- 1 root root    0 Feb 24 15:55 mysecret
drwxr-xr-x 4 root root 4096 Feb 24 15:55 rhsm
COMMIT
--> d0481b33e8f6
d0481b33e8f6d0a33f457fe367070bce68b30e4a67c33d4830d730e2b1abac42

Container file:

FROM python:3.12-slim
RUN --mount=type=secret,id=mysecret  ls -la /run/secrets && cat /run/secrets/mysecret

Output of podman info

Details

Client:
  APIVersion: 5.4.0
  Built: 1739297196
  BuiltTime: Tue Feb 11 19:06:36 2025
  GitCommit: f9f7d48b24b1ca4403f189caaeab1cb8ff4a9aa2
  GoVersion: go1.23.6
  Os: windows
  OsArch: windows/amd64
  Version: 5.4.0
host:
  arch: amd64
  buildahVersion: 1.38.1
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-3.fc41.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 99.05
    systemPercent: 0.3
    userPercent: 0.65
  cpus: 8
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: container
    version: "41"
  eventLogger: journald
  freeLocks: 2041
  hostname: MMD-25394417309
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 5.15.167.4-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: journald
  memFree: 7339290624
  memTotal: 8248123392
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.14.0-1.fc41.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.14.0
    package: netavark-1.14.0-1.fc41.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.14.0
  ociRuntime:
    name: crun
    package: crun-1.20-2.fc41.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.20
      commit: 9c9a76ac11994701dd666c4f0b869ceffb599a66
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20250121.g4f2c8e7-2.fc41.x86_64
    version: |
      pasta 0^20250121.g4f2c8e7-2.fc41.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: unix:///run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: true
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 2147483648
  swapTotal: 2147483648
  uptime: 3h 23m 16.00s (Approximately 0.12 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /home/user/.config/containers/storage.conf
  containerStore:
    number: 5
    paused: 0
    running: 0
    stopped: 5
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/user/.local/share/containers/storage
  graphRootAllocated: 1081101176832
  graphRootUsed: 8067952640
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 101
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 5.3.2
  Built: 1737504000
  BuiltTime: Wed Jan 22 01:00:00 2025
  GitCommit: ""
  GoVersion: go1.23.4
  Os: linux
  OsArch: linux/amd64
  Version: 5.3.2

Similarly as @mpern , specifying src and type just breaks right away:

>podman build --secret=id=mysecret,src=MY_SECRET,type=env -f Containerfile.test . --no-cache 
Error: open MY_SECRET: The system cannot find the file specified.

JarroVGIT avatar Feb 24 '25 16:02 JarroVGIT

Ah, now I think we solved at least the mystery why it works for @hdub-tech: they use a "proper", meaning native, Podman setup.

I forgot to mention that I tested on a M-series macBook, and similarly to WSL2 this means a Podman Machine VM + podman --remote as client.

My semi-educated guess is that the remote client doesn't pass the env var along (-> zero bytes) and for some reason doesn't support the ... type=env syntax at all.(Principle of Least Astonishment, anyone?😅 )

mpern avatar Feb 24 '25 16:02 mpern

I am in the muck with this also on an M3 MBP. The only mechanism I've been able to get working is STDIN.

bigfleet avatar May 21 '25 13:05 bigfleet

The issue with podman-remote was noted here as well: https://github.com/containers/podman/issues/19844

jadejr avatar Jul 16 '25 08:07 jadejr

bump

I am trying to use secret mounts to pass in a token to authenticate to a private artifacts repository from the docker/podman build context

the line in the Dockerfile is

RUN --mount=type=secret,id=UV_INDEX_AZURE_DEVOPS_PASSWORD,env=UV_INDEX_AZURE_DEVOPS_PASSWORD \
    uv export --no-dev --format=requirements-txt --output-file=requirements.txt \
    && uv pip install --system -r requirements.txt

which works perfectly well on Windows using WSL 2 (on Ubuntu 20.04, 22.04, and 24.04) - as well as on Azure Pipeline agents (Ubuntu:latest as of time of writing)

running

 docker build -t reference-api --secret id=UV_INDEX_AZURE_DEVOPS_PASSWORD,env=UV_INDEX_AZURE_DEVOPS_PASSWORD .

with the secret value bound to that variable in my environment produces

Error: building at STEP "RUN --mount=type=secret,id=UV_INDEX_AZURE_DEVOPS_PASSWORD,env=UV_INDEX_AZURE_DEVOPS_PASSWORD uv export --no-dev --format=requirements-txt --output-file=requirements.txt     && uv pip install --system -r requirements.txt": resolving mountpoints for container "55414b07525a95e69b5fbb716be2d86b6b84a96d3454d687e6a0f85f4a683753": secret should have syntax id=id[,target=path,required=bool,mode=uint,uid=uint,gid=uint

zaplapl avatar Oct 15 '25 19:10 zaplapl

Error: building at STEP "RUN --mount=type=secret,id=UV_INDEX_AZURE_DEVOPS_PASSWORD,env=UV_INDEX_AZURE_DEVOPS_PASSWORD uv export --no-dev --format=requirements-txt --output-file=requirements.txt     && uv pip install --system -r requirements.txt": resolving mountpoints for container "55414b07525a95e69b5fbb716be2d86b6b84a96d3454d687e6a0f85f4a683753": secret should have syntax id=id[,target=path,required=bool,mode=uint,uid=uint,gid=uint

Same here, env secrets seem to be broken in Podman. Same Dockerfile works fine in Docker.

uosis avatar Oct 16 '25 18:10 uosis