Kafka - NoBrokerAvailable

[tianhaogu]

Hi, I'm currently working on testing kafka broker / topic and payload for integration testing, I just want to know that to spin up the kafka docker container, I first have to install kafka broker locally right? Because when I run the following code: with KafkaContainer('confluentinc/cp-kafka:5.4.3') as kafka_container: bluhbluhbluh..... a specific kafka gets started, and is waiting to be ready, but then it throws NoBrokerAvailable exception after sine time, so the program stops at the with statement, and the "bluhbluhbluh" part doesn't get executed.

[tillahoffmann]

Hi, could you please post a reproducible example, including the traceback you get?

[tianhaogu]

Note that for the second picture, line 164 in the stack points to line 187 in the code editor (as I added some code later), which is the line of with KafkaContainer('confluentinc/cp-kafka:5.4.3') as kafka_container:.

[tillahoffmann]

Hey, could you please post these as text rather than as screenshots?

[tianhaogu]

After running into the line with KafkaContainer('confluentinc/cp-kafka:5.4.3') as kafka_container:, I met the following exception in the console:

Pulling image confluentinc/cp-kafka:5.4.3
Container started: 2600dbd6a9
Waiting to be ready...
Traceback (most recent call last):
  File "C:/Users/xxx/run_container.py", line 178, in <module>
    main()
  File "C:/Users/xxx/run_container.py", line 164, in main
    with KafkaContainer('confluentinc/cp-kafka:5.4.3') as kafka_container:
  File "C:\Users\xxx\lib\site-packages\testcontainers\core\container.py", line 69, in __enter__
    return self.start()
  File "C:\Users\xxx\lib\site-packages\testcontainers\kafka.py", line 75, in start
    self._connect()
  File "C:\Users\xxx\lib\site-packages\testcontainers\core\waiting_utils.py", line 55, in wrapper
    raise TimeoutException(
testcontainers.core.exceptions.TimeoutException: Wait time (120s) exceeded for _connect(args: (), kwargs {}). Exception: NoBrokersAvailable

[tianhaogu]

Hi, is there any update?

[tillahoffmann]

Which version of the package are you using?

Hi, is there any update?

This project is entirely run by volunteers with other day-time jobs; expect delays in replies accordingly.

[tianhaogu]

Regarding python, it's 3.8. And kafka image version is 5.4.3 as shown above.

[tillahoffmann]

Which version of the testcontainers package are you using?

[tianhaogu]

I just pip install testcontainers[kafka] as followed by the installation instruction.

"docker container ls" shows:

CONTAINER ID   IMAGE                         COMMAND                  CREATED        STATUS        PORTS                               NAMES
2600dbd6a902   confluentinc/cp-kafka:5.4.3   "sh -c 'while [ ! -f…"   38 hours ago   Up 38 hours   9092/tcp, 0.0.0.0:57192->9093/tcp   youthful_panini

after executing with KafkaContainer('confluentinc/cp-kafka:5.4.3') as kafka_container:

[tillahoffmann]

Please run pip freeze and post the output here.

[tianhaogu]

docker==5.0.3 kafka-python==2.0.2 testcontainers==3.6.0