ImageFromDockerfile#withDockerfile can't be used with any other withFile* methods

[rdicroce]

If you use ImageFromDockerfile#withDockerfile, you can't use any other withFile* methods. This happens because ImageFromDockerfile ultimately ends up calling BuildImageCmdImpl#withDockerfile, and that method clearly assumes the specified Dockerfile is the root of the entire context you want to use. It sets a whole bunch of other stuff, including the InputStream. That last part is especially problematic because ImageFromDockerfile calls withDockerfile after it sets the InputStream, so the desired InputStream gets replaced. That, in turn, can lead to hangs if the context is large enough that PipedInputStream blocks waiting for space to become available. See the stack trace below.

I'm not sure if this is supposed to work. DockerignoreTest suggests it should. That test only succeeds because the specified Dockerfile is at the root of the directory passed to withFileFromPath. So if this is supposed to work, it ought to be fixed. If it's not expected to work, then the documentation should say so and ideally ImageFromDockerfile should throw an exception if you try to do it.

Workaround: use withFileFromPath to set the Dockerfile instead.

"main" #1 prio=5 os_prio=0 tid=0x000001cfebce4000 nid=0x386c in Object.wait() [0x00000007ebdfc000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.io.PipedInputStream.awaitSpace(PipedInputStream.java:273)
        at java.io.PipedInputStream.receive(PipedInputStream.java:231)
        - locked <0x0000000775765750> (a java.io.PipedInputStream)
        at java.io.PipedOutputStream.write(PipedOutputStream.java:149)
        at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:253)
        at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:211)
        at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:145)
        - locked <0x0000000776559a58> (a java.util.zip.GZIPOutputStream)
        at org.apache.commons.compress.utils.CountingOutputStream.write(CountingOutputStream.java:48)
        at org.apache.commons.compress.utils.FixedLengthBlockOutputStream$BufferAtATimeOutputChannel.write(FixedLengthBlockOutputStream.java:244)
        at org.apache.commons.compress.utils.FixedLengthBlockOutputStream.writeBlock(FixedLengthBlockOutputStream.java:92)
        at org.apache.commons.compress.utils.FixedLengthBlockOutputStream.maybeFlush(FixedLengthBlockOutputStream.java:86)
        at org.apache.commons.compress.utils.FixedLengthBlockOutputStream.write(FixedLengthBlockOutputStream.java:122)
        at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.write(TarArchiveOutputStream.java:454)
        at java.nio.file.Files.copy(Files.java:2909)
        at java.nio.file.Files.copy(Files.java:3069)
        at org.testcontainers.utility.MountableFile.recursiveTar(MountableFile.java:326)
        at org.testcontainers.utility.MountableFile.recursiveTar(MountableFile.java:335)
        at org.testcontainers.utility.MountableFile.recursiveTar(MountableFile.java:335)
        at org.testcontainers.utility.MountableFile.transferTo(MountableFile.java:300)
        at org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:124)
        at org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:37)
        at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
        - locked <0x00000007753f9790> (a java.util.concurrent.atomic.AtomicReference)
        at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:27)
        at org.testcontainers.shaded.com.google.common.util.concurrent.Futures$3.get(Futures.java:1332)
        at org.testcontainers.images.RemoteDockerImage.getImageName(RemoteDockerImage.java:97)
        at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:56)
        at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:26)
        at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
        - locked <0x00000007754373b8> (a java.util.concurrent.atomic.AtomicReference)
        at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:27)
        at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1277)
        at org.testcontainers.containers.GenericContainer.logger(GenericContainer.java:613)
        at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:320)
        at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:311)
        at com.sg.sciq.testutil.Containers.<init>(Containers.java:71)
        at com.sg.sciq.testutil.Containers.<clinit>(Containers.java:28)
        at com.sg.sciq.testutil.ArquillianContainerGlue$ConfigSetter.resolve(ArquillianContainerGlue.java:17)
        at org.jboss.arquillian.config.impl.extension.ConfigurationRegistrar.loadConfiguration(ConfigurationRegistrar.java:69)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:86)
        at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103)
        at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:90)
        at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:133)
        at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:105)
        at org.jboss.arquillian.core.impl.ManagerImpl.start(ManagerImpl.java:253)
        at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.<init>(EventTestRunnerAdaptor.java:61)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.jboss.arquillian.test.spi.SecurityActions.newInstance(SecurityActions.java:144)
        at org.jboss.arquillian.test.spi.SecurityActions.newInstance(SecurityActions.java:89)
        at org.jboss.arquillian.test.spi.TestRunnerAdaptorBuilder.build(TestRunnerAdaptorBuilder.java:49)
        at org.jboss.arquillian.junit.AdaptorManager.initializeAdaptor(AdaptorManager.java:21)
        at org.jboss.arquillian.junit.AdaptorManagerWithNotifier.initializeAdaptor(AdaptorManagerWithNotifier.java:19)
        at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:109)
        at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:89)
        at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:542)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:770)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:464)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)

[rnorth]

I encountered the same thing earlier this week, by coincidence. I agree that this should not happen, and I'll look into it.

BTW another workaround is to just use withDockerfile, without separately specifying a File/Path for the build context - but only if your Dockerfile is in the build context you need it to be in.

[pvannierop]

I experience the same problem.This does not work for me:

ImageFromDockerfile cbioImage = new ImageFromDockerfile()
            .withFileFromPath(".", RESOURCE_PATH)
            .withDockerFile(DOCKERFILE_AT_DIFFERENT_PATH)

When specifying the withDockerFile the context is set to that dir.

Since my Dockerfile is not at the root of the context you proposed workaround does not work for me:

BTW another workaround is to just use withDockerfile, without separately specifying a File/Path for the build context - but only if your Dockerfile is in the build context you need it to be in.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

[stale[bot]]

This issue has been automatically closed due to inactivity. We apologise if this is still an active problem for you, and would ask you to re-open the issue if this is the case.

[rnorth]

Not stale - reopening.

[kjarosh]

It looks like the same problem as in #3218. I've found this bug when building image with custom Dockerfile path (not in the build context root). I ended up using the deprecated method withDockerfilePath as a workaround (instead of withDockerfile):

new ImageFromDockerfile()
        .withDockerfilePath(RELATIVE_DOCKERFILE_PATH)
        .withFileFromPath("/", CONTEXT_ROOT);

[Gregory-Berkman-Imprivata]

Any progress on this issue?

[adeellis]

Any progress on this?