jib icon indicating copy to clipboard operation
jib copied to clipboard
verified publisher icon GoogleContainerTools

Build fails when using Micronaut

Open pgressa opened this issue 4 years ago • 5 comments

Environment:

  • Jib version: 3.0.0
  • Build tool: maven
  • OS: mac

Description of the issue: When the base image is from public registry hosted by github packages then the build fails. Tested on ghcr.io/graalvm/graalvm-ce:ol8-java11-21.1.0

Expected behavior: Build passes

Steps to reproduce:

jib-maven-plugin Configuration:

            <plugin>
                <groupId>com.google.cloud.tools</groupId>
                <artifactId>jib-maven-plugin</artifactId>
                <version>3.0.0</version>
                <configuration>
                    <to>
                        <image>.....</image>
                    </to>
                    <from>
                        <image>${docker.image.from}</image>
                    </from>
                </configuration>
            </plugin>

Log output:

[INFO] Executing tasks:
[INFO] [======                        ] 18.3% complete
[INFO] > building dependencies layer
[INFO] > pulling base image manifest and container config
May 24, 2021 8:02:52 PM com.google.api.client.http.HttpRequest execute
CONFIG: -------------- REQUEST  --------------
GET https://ghcr.io/v2/graalvm/graalvm-ce/manifests/ol8-java11-21.1.0
Accept: application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json
Accept-Encoding: gzip
User-Agent: jib 3.0.0 jib-maven-plugin Google-HTTP-Java-Client/1.38.1 (gzip)

May 24, 2021 8:02:52 PM com.google.api.client.http.HttpRequest execute
CONFIG: curl -v --compressed -H 'Accept: application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json' -H 'Accept-Encoding: gzip' -H 'User-Agent: jib 3.0.0 jib-maven-plugin Google-HTTP-Java-Client/1.38.1 (gzip)' -- 'https://ghcr.io/v2/graalvm/graalvm-ce/manifests/ol8-java11-21.1.0'
May 24, 2021 8:02:53 PM com.google.api.client.http.HttpResponse <init>
CONFIG: -------------- RESPONSE --------------
HTTP/1.1 401 Unauthorized
Content-Type: application/json
www-authenticate: Bearer realm="https://ghcr.io/token",service="ghcr.io",scope="repository:graalvm/graalvm-ce:pull"
Date: Mon, 24 May 2021 18:02:53 GMT
Content-Length: 73
X-GitHub-Request-Id: E26A:A1C8:4592FD:54CE35:60ABEA4D

May 24, 2021 8:02:53 PM com.google.api.client.util.LoggingByteArrayOutputStream close
[INFO] The base image requires auth. Trying again for ghcr.io/graalvm/graalvm-ce:ol8-java11-21.1.0...
[DEBUG] trying docker-credential-desktop for registry-1.docker.io
[WARNING] The credential helper (docker-credential-desktop) has nothing for server URL: registry-1.docker.io
[WARNING] 
Got output:

credentials not found in native keychain

[DEBUG] trying docker-credential-desktop for registry.hub.docker.com
[WARNING] The credential helper (docker-credential-desktop) has nothing for server URL: registry.hub.docker.com
[WARNING] 
Got output:

credentials not found in native keychain

[DEBUG] trying docker-credential-desktop for index.docker.io
[INFO] Using credentials from Docker config (/Users/pgressa/.docker/config.json) for openjdk:16-alpine
[DEBUG] WWW-Authenticate for ghcr.io/graalvm/graalvm-ce:ol8-java11-21.1.0: Bearer realm="https://ghcr.io/token",service="ghcr.io",scope="repository:graalvm/graalvm-ce:pull"
[INFO] Executing tasks:
[INFO] [=======                       ] 23.3% complete
[INFO] > pulling base image manifest
[INFO] > building dependencies layer
May 24, 2021 8:02:53 PM com.google.api.client.http.HttpRequest execute
CONFIG: -------------- REQUEST  --------------
GET https://ghcr.io/token?service=ghcr.io&scope=repository:graalvm/graalvm-ce:pull
Accept: */*
Accept-Encoding: gzip
Authorization: <Not Logged>
User-Agent: jib 3.0.0 jib-maven-plugin Google-HTTP-Java-Client/1.38.1 (gzip)

May 24, 2021 8:02:53 PM com.google.api.client.http.HttpRequest execute
CONFIG: curl -v --compressed -H 'Accept: */*' -H 'Accept-Encoding: gzip' -H 'Authorization: <Not Logged>' -H 'User-Agent: jib 3.0.0 jib-maven-plugin Google-HTTP-Java-Client/1.38.1 (gzip)' -- 'https://ghcr.io/token?service=ghcr.io&scope=repository:graalvm/graalvm-ce:pull'
May 24, 2021 8:02:53 PM com.google.api.client.http.HttpResponse <init>
CONFIG: -------------- RESPONSE --------------
HTTP/1.1 403 Forbidden
Content-Type: application/json
docker-distribution-api-version: registry/2.0
Date: Mon, 24 May 2021 18:02:53 GMT
Content-Length: 50
X-GitHub-Request-Id: E26B:A1C8:459308:54CE42:60ABEA4D

May 24, 2021 8:02:53 PM com.google.api.client.util.LoggingByteArrayOutputStream close
[DEBUG] TIMED   Pulling base image manifest : 1467.481 ms
[DEBUG] TIMED   Building image to Docker daemon : 1497.551 ms
[INFO] Executing tasks:
[INFO] [==========                    ] 33.3% complete
[INFO] > building dependencies layer
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  8.047 s
[INFO] Finished at: 2021-05-24T20:02:53+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.0.0:dockerBuild (default-dockerBuild) on project carts: Build to Docker daemon failed, perhaps you should make sure you have permissions for ghcr.io/graalvm/graalvm-ce and set correct credentials. See https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#what-should-i-do-when-the-registry-responds-with-forbidden-or-denied for help: Unauthorized for ghcr.io/graalvm/graalvm-ce: 403 Forbidden
[ERROR] GET https://ghcr.io/token?service=ghcr.io&scope=repository:graalvm/graalvm-ce:pull
[ERROR] {"errors":[{"code":"DENIED","message":"denied"}]}
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.0.0:dockerBuild (default-dockerBuild) on project carts: Build to Docker daemon failed, perhaps you should make sure you have permissions for ghcr.io/graalvm/graalvm-ce and set correct credentials. See https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#what-should-i-do-when-the-registry-responds-with-forbidden-or-denied for help
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:39)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:122)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:61)
Caused by: org.apache.maven.plugin.MojoExecutionException: Build to Docker daemon failed, perhaps you should make sure you have permissions for ghcr.io/graalvm/graalvm-ce and set correct credentials. See https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#what-should-i-do-when-the-registry-responds-with-forbidden-or-denied for help
    at com.google.cloud.tools.jib.maven.BuildDockerMojo.execute (BuildDockerMojo.java:166)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:39)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:122)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:61)
Caused by: com.google.cloud.tools.jib.api.RegistryUnauthorizedException: Unauthorized for ghcr.io/graalvm/graalvm-ce
    at com.google.cloud.tools.jib.plugins.common.JibBuildRunner.runBuild (JibBuildRunner.java:265)
    at com.google.cloud.tools.jib.maven.BuildDockerMojo.execute (BuildDockerMojo.java:102)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:39)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:122)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:61)
Caused by: com.google.cloud.tools.jib.http.ResponseException: 403 Forbidden
GET https://ghcr.io/token?service=ghcr.io&scope=repository:graalvm/graalvm-ce:pull
{"errors":[{"code":"DENIED","message":"denied"}]}

    at com.google.cloud.tools.jib.http.FailoverHttpClient.call (FailoverHttpClient.java:329)
    at com.google.cloud.tools.jib.http.FailoverHttpClient.call (FailoverHttpClient.java:250)
    at com.google.cloud.tools.jib.registry.RegistryAuthenticator.authenticate (RegistryAuthenticator.java:283)
    at com.google.cloud.tools.jib.registry.RegistryAuthenticator.authenticate (RegistryAuthenticator.java:257)
    at com.google.cloud.tools.jib.registry.RegistryAuthenticator.authenticatePull (RegistryAuthenticator.java:176)
    at com.google.cloud.tools.jib.registry.RegistryClient.doBearerAuth (RegistryClient.java:336)
    at com.google.cloud.tools.jib.registry.RegistryClient.authPullByWwwAuthenticate (RegistryClient.java:395)
    at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call (PullBaseImageStep.java:177)
    at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call (PullBaseImageStep.java:69)
    at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly (TrustedListenableFutureTask.java:125)
    at com.google.common.util.concurrent.InterruptibleTask.run (InterruptibleTask.java:69)
    at com.google.common.util.concurrent.TrustedListenableFutureTask.run (TrustedListenableFutureTask.java:78)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:834)
Caused by: com.google.api.client.http.HttpResponseException: 403 Forbidden
GET https://ghcr.io/token?service=ghcr.io&scope=repository:graalvm/graalvm-ce:pull
{"errors":[{"code":"DENIED","message":"denied"}]}

    at com.google.api.client.http.HttpRequest.execute (HttpRequest.java:1113)
    at com.google.cloud.tools.jib.http.FailoverHttpClient.call (FailoverHttpClient.java:323)
    at com.google.cloud.tools.jib.http.FailoverHttpClient.call (FailoverHttpClient.java:250)
    at com.google.cloud.tools.jib.registry.RegistryAuthenticator.authenticate (RegistryAuthenticator.java:283)
    at com.google.cloud.tools.jib.registry.RegistryAuthenticator.authenticate (RegistryAuthenticator.java:257)
    at com.google.cloud.tools.jib.registry.RegistryAuthenticator.authenticatePull (RegistryAuthenticator.java:176)
    at com.google.cloud.tools.jib.registry.RegistryClient.doBearerAuth (RegistryClient.java:336)
    at com.google.cloud.tools.jib.registry.RegistryClient.authPullByWwwAuthenticate (RegistryClient.java:395)
    at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call (PullBaseImageStep.java:177)
    at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call (PullBaseImageStep.java:69)
    at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly (TrustedListenableFutureTask.java:125)
    at com.google.common.util.concurrent.InterruptibleTask.run (InterruptibleTask.java:69)
    at com.google.common.util.concurrent.TrustedListenableFutureTask.run (TrustedListenableFutureTask.java:78)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:834)
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Additional Information:

The first call that tries to download manifest: curl -v --compressed -H 'Accept: application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json' -H 'Accept-Encoding: gzip' -H 'User-Agent: jib 3.0.0 jib-maven-plugin Google-HTTP-Java-Client/1.38.1 (gzip)' -- 'https://ghcr.io/v2/graalvm/graalvm-ce/manifests/ol8-java11-21.1.0' fails however when the Auth header -H 'Authorization: <Not Logged>' is added then if passes and manifest is downloaded.

pgressa avatar May 24 '21 18:05 pgressa

We know that public ghcr.io certainly works.

$ git clone https://github.com/GoogleContainerTools/jib.git
$ cd jib/examples/helloworld/
$ mvn compile
$ mvn -Djib.from.image=ghcr.io/graalvm/graalvm-ce:ol8-java11-21.1.0 -Djib.to.image=test jib:dockerBuild
...
[INFO] --- jib-maven-plugin:3.0.0:dockerBuild (default-cli) @ helloworld ---
...
[WARNING] Base image 'ghcr.io/graalvm/graalvm-ce:ol8-java11-21.1.0' does not use a specific image digest - build may not be reproducible
[INFO] The base image requires auth. Trying again for ghcr.io/graalvm/graalvm-ce:ol8-java11-21.1.0...
[INFO] Using base image with digest: sha256:04bd41711712997222a9d16c9e5feeaf27c6550e6e6bd084fd7b59519d1bcdf6
...
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  5.525 s
[INFO] Finished at: 2021-05-24T17:15:39-04:00

Can you follow the steps above carefully and tell us if it works?

And did you edit your Jib log while copying-and-pasting? It doesn't make sense at all that there's a reference to a Docker Hub image openjdk:16-alpine.

[INFO] Using credentials from Docker config (/Users/pgressa/.docker/config.json) for openjdk:16-alpine

Where do you think it's coming from?

chanseokoh avatar May 24 '21 21:05 chanseokoh

And are you using any Jib extension?

chanseokoh avatar May 24 '21 21:05 chanseokoh

@chanseokoh yeah, I'm using micronaut maven plugin, that has jib extension. Yes, when I tried your steps, it works. So I'll look into micronaut maven plugin. Thanks @chanseokoh !

pgressa avatar May 25 '21 08:05 pgressa

I remember a chat with a Micronaut dev where they said their Jib extension changes a base image to openjdk. (There's a related Jib issue #2953.) Until Jib can resolve #2953 (which is tricky to fix due to how the Jib infrastructure processes configurations), the Micronaut Dev said they could implement a workaround for that. But in your case, the extension doesn't really seem to have changed your base image to openjdk, but it looks like it re-configured the auth and resulted in fetching credentials for openjdk (i.e., for Docker Hub).

I think an immediate workaround is to do docker logout, so give it a try. The actual error I think is that Jib picks up credentials for Docker Hub and use them when authenticating with ghcr.io.

chanseokoh avatar May 25 '21 14:05 chanseokoh

AFAIK this is still an issue, but we do not have immediate plans to work on this. Any community contributions are welcome.

alicejli avatar Sep 06 '23 16:09 alicejli