TestContainers Kafka wrapper fails to start Confleunt Kafka

[yaraskm]

trafficstars

The Test Containers - Kafka module fails to startup a container. The following starter example works fine on a standard Docker backend but fails with Kubedock.

import org.junit.jupiter.api.Test;

import org.testcontainers.containers.KafkaContainer;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.DockerImageName;

@Testcontainers
public class KafkaTest {

    @Test
    void testKafkaStartup() {
        KafkaContainer KAFKA_CONTAINER = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:5.5.0"))
                .withStartupAttempts(3);

        KAFKA_CONTAINER.start();
        KAFKA_CONTAINER.stop();
    }
}

Running this example against a local Kubedock build from master yields:

[GIN] 2022/02/14 - 12:33:51 | 201 |       165.8µs |       127.0.0.1 | POST     "/containers/create"
[GIN] 2022/02/14 - 12:33:53 | 204 |    2.4264976s |       127.0.0.1 | POST     "/containers/44e0da1c4a4ff1260dfe20404f1a9f916c88ab78dcf1f0d0204a51fc228c0cde/start"
I0214 12:33:53.722604   27595 portforward.go:42] start port-forward 37943->9093
I0214 12:33:53.722665   27595 portforward.go:42] start port-forward 52796->2181
[GIN] 2022/02/14 - 12:33:53 | 200 |        86.8µs |       127.0.0.1 | GET      "/containers/44e0da1c4a4ff1260dfe20404f1a9f916c88ab78dcf1f0d0204a51fc228c0cde/json"
I0214 12:33:53.833324   27595 copy.go:36] copy 2048 bytes to 44e0da1c4a4f:/
E0214 12:33:58.908660   27595 v2.go:168] io: read/write on closed pipe
E0214 12:33:58.914197   27595 util.go:18] error during request[500]: command terminated with exit code 2
[GIN] 2022/02/14 - 12:33:58 | 500 |    5.1302852s |       127.0.0.1 | PUT      "/containers/44e0da1c4a4ff1260dfe20404f1a9f916c88ab78dcf1f0d0204a51fc228c0cde/archive?noOverwriteDirNonDir=false&path=%2F"
[GIN] 2022/02/14 - 12:33:59 | 200 |    102.6893ms |       127.0.0.1 | GET      "/containers/44e0da1c4a4ff1260dfe20404f1a9f916c88ab78dcf1f0d0204a51fc228c0cde/logs?stdout=true&stderr=true&since=0"
[GIN] 2022/02/14 - 12:33:59 | 201 |       144.7µs |       127.0.0.1 | POST     "/containers/create"
[GIN] 2022/02/14 - 12:34:01 | 204 |     2.379796s |       127.0.0.1 | POST     "/containers/86abe9fd787d0b557bde2496a269a708f03a9a368b05989116d6cfad93ca41e3/start"
I0214 12:34:01.436944   27595 portforward.go:42] start port-forward 38007->2181
I0214 12:34:01.437094   27595 portforward.go:42] start port-forward 52166->9093
[GIN] 2022/02/14 - 12:34:01 | 200 |       175.1µs |       127.0.0.1 | GET      "/containers/86abe9fd787d0b557bde2496a269a708f03a9a368b05989116d6cfad93ca41e3/json"
I0214 12:34:01.491702   27595 copy.go:36] copy 2048 bytes to 86abe9fd787d:/
E0214 12:34:01.989394   27595 v2.go:168] io: read/write on closed pipe
E0214 12:34:01.990213   27595 util.go:18] error during request[500]: command terminated with exit code 2
[GIN] 2022/02/14 - 12:34:01 | 500 |    547.6007ms |       127.0.0.1 | PUT      "/containers/86abe9fd787d0b557bde2496a269a708f03a9a368b05989116d6cfad93ca41e3/archive?noOverwriteDirNonDir=false&path=%2F"
[GIN] 2022/02/14 - 12:34:02 | 200 |    103.6247ms |       127.0.0.1 | GET      "/containers/86abe9fd787d0b557bde2496a269a708f03a9a368b05989116d6cfad93ca41e3/logs?stdout=true&stderr=true&since=0"
[GIN] 2022/02/14 - 12:34:02 | 201 |       183.3µs |       127.0.0.1 | POST     "/containers/create"
[GIN] 2022/02/14 - 12:34:04 | 204 |    2.4081256s |       127.0.0.1 | POST     "/containers/479e20e215dd43c14171afb588470b239a31c475f8ac44670d54b95dad71c6fd/start"
I0214 12:34:04.518805   27595 portforward.go:42] start port-forward 44148->9093
I0214 12:34:04.518965   27595 portforward.go:42] start port-forward 47341->2181
[GIN] 2022/02/14 - 12:34:04 | 200 |       207.4µs |       127.0.0.1 | GET      "/containers/479e20e215dd43c14171afb588470b239a31c475f8ac44670d54b95dad71c6fd/json"
I0214 12:34:04.573697   27595 copy.go:36] copy 2048 bytes to 479e20e215dd:/
E0214 12:34:05.047982   27595 v2.go:168] io: read/write on closed pipe
E0214 12:34:05.052508   27595 util.go:18] error during request[500]: command terminated with exit code 2
[GIN] 2022/02/14 - 12:34:05 | 500 |    526.6571ms |       127.0.0.1 | PUT      "/containers/479e20e215dd43c14171afb588470b239a31c475f8ac44670d54b95dad71c6fd/archive?noOverwriteDirNonDir=false&path=%2F"
[GIN] 2022/02/14 - 12:34:05 | 200 |    101.7087ms |       127.0.0.1 | GET      "/containers/479e20e215dd43c14171afb588470b239a31c475f8ac44670d54b95dad71c6fd/logs?stdout=true&stderr=true&since=0"

I've looked at this off and on over the last few weeks and was hopeful that perhaps the fix for #6 would resolve this as well.

[joyrex2001]

Locally using minikube the test doesn't fail for me. Are you using OpenShift?

Upping the verbosity (e.g. kubedock server -P -v 5 --port-forward) displays what the startup actually does; it rewrites the entrypoint to watch for a certain shell script to be present, and the copy actually copies that specific shell script to the container. After that, it does a local nc to see if the something is listening to the zookeeper/kafka ports.

[yaraskm]

Good point, I'm using vanilla Kubernetes but setting a different user to run as (1001). I suspect this is an issue with the image itself then, will confirm by running as the user expected in the image. Thanks!

[joyrex2001]

When running kubedock to enforce user 1001, the test fails indeed. This is because it tries to copy a shell script to /, which is not allowed by that user. The issue is similar to: https://github.com/quarkusio/quarkus/pull/19736

[joyrex2001]

Closing in favour of #43.