compose Compose v2 build doesn't respect local git credential helpers

Description

Steps to reproduce the issue:

Create a private git repository, e.g. at GitHub, with a valid Dockerfile in it.
Define a git credential helper for accessing the repository.

Create a Compose file with the build field referring to the private repository; e.g.

services:
  my-service:
    build: https://github.com/username/private-repo.git#main

docker compose up

Describe the results you received:

[+] Building 1.1s (1/1) FINISHED                                                                                                                                                       
 => ERROR [internal] load git source https://github.com/username/private-repo.git#main                                                                                                 1.1s
------
 > [internal] load git source https://github.com/username/private-repo.git#main:
#0 0.021 hint: Using 'master' as the name for the initial branch. This default branch name
#0 0.021 hint: is subject to change. To configure the initial branch name to use in all
#0 0.021 hint: of your new repositories, which will suppress this warning, call:
#0 0.021 hint: 
#0 0.021 hint:  git config --global init.defaultBranch <name>
#0 0.021 hint: 
#0 0.021 hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
#0 0.021 hint: 'development'. The just-created branch can be renamed via this command:
#0 0.021 hint: 
#0 0.021 hint:  git branch -m <name>
#0 0.021 Initialized empty Git repository in /var/lib/docker/overlay2/93j3bj8v2kg85qlagdlqhj8cl/diff/
#0 1.122 fatal: could not read Username for 'https://github.com': terminal prompts disabled
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to load cache key: failed to fetch remote https://github.com/username/private-repo.git: exit status 128

Process finished with exit code 17

Describe the results you expected:

docker image build https://github.com/username/private-repo.git#main would succeed (uses credential helpers); therefore, expecting Docker Compose to use credential helpers, clone the private repository, build the image, and run it.

This issue was once brought up in #5854, and fixed in Docker Compose v1.

Output of docker compose version:

Docker Compose version v2.6.1

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.6.1)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 20
  Running: 0
  Paused: 0
  Stopped: 20
 Images: 168
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.0-40-generic
 Operating System: Ubuntu 22.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.41GiB
 Name: my-laptop
 ID: VEH6:NI7F:DGF5:PZWB:AYBC:LOUL:CXTY:4BDT:CG31:YCXG:DFX6:MK6G
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Jun 27 '22 20:06 agcom

I am facing the same issue.

Oct 09 '22 16:10 reeshabhranjan

Same issue here !

Dec 24 '22 11:12 Amphaal

Same issue! Any help would be appreciated!

Feb 02 '23 19:02 sabinamiani

Can you please confirm you don't get the same issue using plain docker buildx build https://github.com/... command ? You can force compose to run the "classic" builder by setting DOCKER_BUILDKIT=0

Feb 06 '23 15:02 ndeloof

I ran a quick test:

I created a private git repository set on gitlab.com
I created a personal access token
configured git to cache this access token on disk, and cloned once the repository

~/.gitconfig

[credential "https://gitlab.com"]
	username = ndeloof
[credential]
	helper = store

~/.git-credentials

https://ndeloof:********************@gitlab.com

With this setup, I can run a build with the classic builder, but not with buildkit enabled:

$ DOCKER_BUILDKIT=1 docker build https://gitlab.com/ndeloof/foobar.git

[+] Building 0.5s (1/1) FINISHED                                                                                                            
 => ERROR [internal] load git source https://gitlab.com/ndeloof/foobar.git                                                             0.5s
------                                                                                                                                      
 > [internal] load git source https://gitlab.com/ndeloof/foobar.git:
#1 0.012 Initialized empty Git repository in /var/lib/docker/overlay2/id9iida59629zmwzbvsv80aho/diff/
#1 0.459 fatal: could not read Username for 'https://gitlab.com': terminal prompts disabled
------
failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to load cache key: failed to fetch remote https://gitlab.com/ndeloof/foobar.git: exit status 128


$ DOCKER_BUILDKIT=0 docker build https://gitlab.com/ndeloof/foobar.git
Sending build context to Docker daemon  69.63kB
Step 1/1 : FROM alpine
 ---> d74e625d9115
Successfully built d74e625d9115

So, this is a buildx limitation. Workaround is to run compose with classic builder selected (DOCKER_BUILDKIT=0) I haven't found a matching issue reported on https://github.com/docker/buildx

cc @crazy-max

Feb 17 '23 14:02 ndeloof