Compose fails to use registry credentials

[birla]

Bug description

Compose extension (which uses github.com/docker/compose) is not able to access registry credentials configured in podman.

Quick fix: pull all required images manually, using podman pull privateregistry.com/image:tag then run podman compose up and it works because the local image cache is seeded now.

Operating system

Windows 11

Installation Method

Installer from website/GitHub releases

Version

1.9.0

Steps to reproduce

1. Install Podman Desktop and dependencies.
2. Setup Compose extension (system-wide) using docker compose.
3. Login into a private container registry using either podman login or using the Podman desktop UI.
4. Verify that connection to the private registry is working by pulling an image using podman pull or the Podman desktop UI.
5. Try to spin up a docker compose file which relies on any image from the private container registry, not already available locally. podman compose up

Relevant log output

Error response from daemon: {"message":"unable to retrieve auth token: invalid username/password: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information."}
Error: executing C:\Users\username\AppData\Local\Microsoft\WindowsApps\docker-compose.exe -f .\docker-compose.yml up -d: exit status 18

Additional context

No response

[cdrage]

Bug description

Compose extension (which uses github.com/docker/compose) is not able to access registry credentials configured in podman.

Quick fix: pull all required images manually, using podman pull privateregistry.com/image:tag then run podman compose up and it works because the local image cache is seeded now.

Operating system

Windows 11

Installation Method

Installer from website/GitHub releases

Version

1.9.0

Steps to reproduce

1. Install Podman Desktop and dependencies.
2. Setup Compose extension (system-wide) using docker compose.
3. Login into a private container registry using either podman login or using the Podman desktop UI.
4. Verify that connection to the private registry is working by pulling an image using podman pull or the Podman desktop UI.
5. Try to spin up a docker compose file which relies on any image from the private container registry, not already available locally. podman compose up

Relevant log output

Error response from daemon: {"message":"unable to retrieve auth token: invalid username/password: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information."}
Error: executing C:\Users\username\AppData\Local\Microsoft\WindowsApps\docker-compose.exe -f .\docker-compose.yml up -d: exit status 18

Additional context

No response

Hi @birla !

TLDR; Delete /.docker/config.json and try again.

I'm unable to reproduce the exact issue, but I have a feeling that it's because of your ~/.docker/config.json configuration that may be getting credentials from an older Docker Desktop install (specifically desktop). If you have any information in ~/.docker/config.json it will override that compose information.

I actually encountered a different issue which made me look at https://stackoverflow.com/questions/67642620/docker-credential-desktop-not-installed-or-not-available-in-path

After I deleted ~/.docker/config.json I was able to pull from a private registry.

This was the compose file I used:

services:

  redis-leader:
    container_name: redis-leader
    image: redis:latest
    ports:
      - "6379"

  redis-replica:
    container_name: redis-replica
    image: redis:latest
    ports:
      - "6379"
    command: redis-server --replicaof redis-leader 6379

  web:
    container_name: web
    image: docker.io/cdrage/foobar
    ports:
      - "8080:8080"

Note that cdrage/foobar is a private repo and I had to do podman login before doing podman compose up.

Let me know if this fixes your issue.

[birla]

Hi @cdrage!

Thank you for getting back. As you correctly deduced, I did have Docker Desktop installed before my foray with Podman Desktop.

I also faced the same issue with docker-credential-desktop you mentioned and edited ~/.docker/config.json as suggested by the community.

Today, I tried removing ~/.docker/config.json and subsequently the folder ~/.docker, but I am still facing the same issue. I did get the "Login Succeeded!" message every time from podman login.

Let me know if I can provide further diagnosis / log files from my system to reproduce this. My team and I are all facing this issue.

[cdrage]

Hi @cdrage!

Thank you for getting back. As you correctly deduced, I did have Docker Desktop installed before my foray with Podman Desktop.

I also faced the same issue with docker-credential-desktop you mentioned and edited ~/.docker/config.json as suggested by the community.

Today, I tried removing ~/.docker/config.json and subsequently the folder ~/.docker, but I am still facing the same issue. I did get the "Login Succeeded!" message every time from podman login.

Let me know if I can provide further diagnosis / log files from my system to reproduce this. My team and I are all facing this issue.

Is there any way you can tell me which private registry? Is it an internal registry / self hosted or cloud?

Trying to replicate this against quay.io / docker hub I'm unable to reproduce.

[birla]

Hi @cdrage!

Thank you for getting back. As you correctly deduced, I did have Docker Desktop installed before my foray with Podman Desktop.

I also faced the same issue with docker-credential-desktop you mentioned and edited ~/.docker/config.json as suggested by the community.

Today, I tried removing ~/.docker/config.json and subsequently the folder ~/.docker, but I am still facing the same issue. I did get the "Login Succeeded!" message every time from podman login.

Let me know if I can provide further diagnosis / log files from my system to reproduce this. My team and I are all facing this issue.

Is there any way you can tell me which private registry? Is it an internal registry / self hosted or cloud?

Trying to replicate this against quay.io / docker hub I'm unable to reproduce.

I am using Azure Container Registry, will try with a public one and get back to you.

[birla]

@cdrage, tried with Docker Hub, using a private image, and a docker compose file. I am able to reproduce the issue, below is the log:

C:\Projects\rnd\docker-private-repo-test>podman login  -u username docker.io
Password: 
Login Succeeded!

C:\Projects\rnd\docker-private-repo-test>podman compose up
>>>> Executing external compose provider "C:\\Program Files\\RedHat\\Podman\\docker-compose.exe". Please refer to the documentation for details. <<<<

[+] Running 1/1
✘ node-app Error {"message":"denied: requested access to the resource is denied"}                                                                                           5.8s 
Error response from daemon: {"message":"denied: requested access to the resource is denied"}
Error: executing C:\Program Files\RedHat\Podman\docker-compose.exe up: exit status 18

Subsequently, pulling the image directly seems to work:

C:\Projects\rnd\docker-private-repo-test>podman pull username/docker-private-repo-test:latest
Resolving "username/docker-private-repo-test" using unqualified-search registries (/etc/containers/registries.conf.d/999-podman-machine.conf)
Trying to pull docker.io/username/docker-private-repo-test:latest...
Getting image source signatures
Copying blob sha256:45a0166cf96b2a4f328191f78f73e68e0e340450a962ff6fc34013111c014d24
Copying blob sha256:e5d288a133c72d9c56a2071dc40223467a08d04247c74229969a13f5076cedeg
Copying blob sha256:832e0dc1fe41d061d47d41e00abf6a9dab0c399d69bae854ef1bffe1976c2df1
Copying blob sha256:8ae971f79f99381da4a83f2cb63aa502fb847cc81a2f270326753f6289562dfd
Copying blob sha256:de95a267926c644e0a631e15c17f37176e2418885b6bbf84a873a7dde3877068
Copying blob sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf9
Copying blob sha256:2c6e675058a6190797a63111833f490d76d33516fcc5e40f2657edd16fba3447
Copying config sha256:7abd4b954cd34efe1cd0ee341ae20db823d9eefffdeb12abad39332e866aeb86
Writing manifest to image destination
6e9b9fe4de1b0e6d01fee20ca96492232f5d5b1b01bfe78d9d840bda6c858afa

[birla]

@cdrage , just installed podman-compose, and it was able to pull from both Docker Hub and Azure Container Registry without any issue using the command podman-compose up

[cdrage]

@cdrage , just installed podman-compose, and it was able to pull from both Docker Hub and Azure Container Registry without any issue using the command podman-compose up

Hi @birla still unable to re-produce :( at this point though, I believe this is actually a podman issue since this reflects directly with the CLI.

I'm going to transfer this issue to https://github.com/containers/podman if that's okay with you.

[gautaz]

Hello, I have the exact same issue. I also had a ~/.docker/config.json file with credentials from a previous docker install. Removing this file did also not fix the issue. So for now I am working around this the same way as @birla.

I got this error after a podman system reset, I have no idea if doing this might reproduce the issue... I operated this reset because of inconsistencies in ~/.local/share/containers subsequent to a crash and I had to hard reboot.

[birla]

Everyone on my team is facing the same issue, we used docker desktop until the company asked us to migrate to podman instead due to licensing issues.

[rrmistry]

Eventually got this working by copying ~/.config/containers/auth.json to ~/.docker/config.json.

Not ideal and would like a reasonable solution.

[birla]

Eventually got this working by copying ~/.config/containers/auth.json to ~/.docker/config.json.

Not ideal and would like a reasonable solution.

I can confirm, this worked for me too.

[rhatdan]

No ideal way to solve this.

[jakubsemerak]

We are facing the same issue in our company, the workaround from @rrmistry seems to work, but it is not ideal.

[empire-penguin]

Eventually got this working by copying ~/.config/containers/auth.json to ~/.docker/config.json.

Not ideal and would like a reasonable solution.

We faced a similar problem, and I can confirm this solution works. However we wanted this to apply systemwide and not have each user do this. Here is what worked for us.

cp ~/.config/containers/auth.json /etc/docker/config.json
export DOCKER_CONFIG=/etc/docker

We put the environment variable in /etc/profile.d/. This makes sure that all users have the environment variable set.

Then you can run docker compose build <SERVICE>.

[novotnyJiri]

Encountered the issue as well. Kudos to @rrmistry for the workaround. Simply works.

[gautaz]

OK, there is no ~/.config/containers/auth.json in my environment (NixOS).

Reviewing the podman-login manpage, the right path is ${XDG_RUNTIME_DIR}/containers/auth.json. So a ln -s ${XDG_RUNTIME_DIR}/containers/auth.json ~/.docker/config.json did the trick for me.

Thanks @rrmistry.

[CarstenLeue]

Suggestion https://github.com/containers/podman/issues/22682#issuecomment-2216754294 is working for me, too, but I was hoping for a podman only solution

@rhatdan

[rhatdan]

With Podman login you can tell it to put the auth.json file in ~/.docker/config.json.

I guess when using podman compose we could check if the link exists and if not create it. But not much else we can do.

[dmurvihill]

Please advise the correct file path to delete on Windows. Thx.

[jacob-kenin-elucid]

I am running Ubuntu 24.04 starting from a clean install with just the docker engine. Based off the findings in https://github.com/containers/podman/issues/22682#issuecomment-2367652052 I did the following:

mkdir ~/.docker
ln -s ${XDG_RUNTIME_DIR}/containers/auth.json ~/.docker/config.json

And now I can successfully pull containers from GitHub Container Registry (GHCR) after authenticating.

[mfontana-elem]

With Podman login you can tell it to put the auth.json file in ~/.docker/config.json.

I guess when using podman compose we could check if the link exists and if not create it. But not much else we can do.

Thanks for the answer. Can you explain, for us who don't know, why a proper solution can't be implemented by podman?

Is podman compose just calling docker compose with some plumbing to make the latter run using podman as the container manager? This is the only scenario I can imagine right know for which a non-hacky solution would be impossible.

[prakharbirla-ng]

Thanks for the answer. Can you explain, for us who don't know, why a proper solution can't be implemented by podman?

Is podman compose just calling docker compose with some plumbing to make the latter run using podman as the container manager? This is the only scenario I can imagine right know for which a non-hacky solution would be impossible.

Yes, your assumption is correct. Podman uses an external provider for compose commands wiz docker compose.

[mattnathan]

As this ticket is where you end up after searching for podman compose authentication required I thought I'd mention my specific setup which isn't listed in any of the above posts.

I'm running on MacOS, have never installed Docker or Docker Desktop. I installed Podman via the Podman Desktop installer and am using default settings which uses docker-compose as the compose tool.

I've run podman login which succeeded. My private registry is AWS ECR.

The podman-login man page mentions that on mac the auth.json file is located in $HOME/.config/containers/auth.json. I have no ~/.docker/ folder. XDG_RUNTIME_DIR is not set.

For me on macOS this made the issue go away (thanks to previous posts):

mkdir ~/.docker/
ln -s ~/.config/containers/auth.json ~/.docker/config.json

Hopefully this kind of thing will make its way into the podman cli at some point.

[ljouon]

Thanks @mattnathan. That worked for me as well.

[whyayala]

OK, there is no ~/.config/containers/auth.json in my environment (NixOS).

Reviewing the podman-login manpage, the right path is ${XDG_RUNTIME_DIR}/containers/auth.json. So a ln -s ${XDG_RUNTIME_DIR}/containers/auth.json ~/.docker/config.json did the trick for me.

Thanks @rrmistry.

I'm on macOs and this is close to the solution I arrived at (ln -s ~/.config/containers/auth.json ~/.docker/config.json) before coming here to create an issue and finding this thread. Symlinking is probably the least encumber-some solution if you just want to fix your machine and not have to worry about it again.

As others have stated, the reason this happens is because podman login stores credentials in one place and the compose extension in podman desktop is just a wrapper for docker compose which looks for credentials in another location.

This was very annoying to troubleshoot and probably turns people off to podman.