connection refused for kafka transport

[Nahel88]

Hi,

I am having this issue with the code while trying to use : confluentinc/cp-kafka

Error :

time=2025-10-01T05:35:12.167Z level=ERROR msg="error transporter" error="kafka: client has run out of available brokers to talk to: dial tcp 172.20.0.3:9092: connect: connection refused for kafka transport"

docker-compose. File:

`services:
  kafka:
    image: confluentinc/cp-kafka:latest
    ports:
      - 9092:9092
    environment:

      KAFKA_KRAFT_MODE: "true"  # Enables KRaft mode.
      KAFKA_PROCESS_ROLES: controller,broker  # Kafka acts as both controller and broker.
      KAFKA_NODE_ID: 1  # Unique ID for the Kafka instance.
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
      KAFKA_ADVERTISED_LISTENERS: "BROKER://kafka:9092"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT,IB:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: BROKER
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LOG_DIRS: /var/lib/kafka/data  # Log storage location.
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"  # Enables automatic topic creation.
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1  # Single replica for simplicity.
      KAFKA_LOG_RETENTION_HOURS: 168  # Log retention period (7 days).
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0  # No rebalance delay.
      CLUSTER_ID: AAAAAAAAAAAAAAAAAAAAAA
      KAFKA_ENABLE_KRAFT: yes


        # dd if=/dev/zero of=/dev/stdout count=16 bs=1 | openssl enc -A -a


        # head -c 20 /dev/zero | base64
    restart: always
  grafana:
    image: grafana/grafana:12.1
    environment:
      - GF_INSTALL_PLUGINS=vertamedia-clickhouse-datasource
    #  - GF_INSTALL_PLUGINS=grafana-clickhouse-datasource
    #  - GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=vertamedia-clickhouse-datasource
    ports:
      - 3000:3000
    restart: always
    volumes:
      - ./grafana/datasources-ch.yml:/etc/grafana/provisioning/datasources/datasources-ch.yml
      - ./grafana/dashboards.yml:/etc/grafana/provisioning/dashboards/dashboards.yml
      - ./grafana/dashboards:/var/lib/grafana/dashboards
  prometheus:
    image: prom/prometheus:v3.6.0
    ports:
      - 9090:9090
    restart: always
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
  goflow2:
    build:
      context: ../../
      dockerfile: Dockerfile
      args:
        VERSION: compose
        LDFLAGS: -X main.version=compose
    image: netsampler/goflow2
    depends_on:
      - kafka
    ports:
      - 8080:8080
      - 6343:6343/udp
      - 2055:2055/udp
    restart: always
    command:
      - -transport.kafka.brokers=kafka:9092
      - -transport=kafka
      - -transport.kafka.topic=flows
      - -format=bin
  db:
    image: clickhouse/clickhouse-server:24.11.1.2557-alpine
    ports:
      - 8123:8123
    volumes:
      - ./clickhouse:/docker-entrypoint-initdb.d/
      #- ../../pb/flow.proto:/var/lib/clickhouse/format_schemas/flow.proto
      # the following files have 101:101 as owner
      - ./clickhouse/flow.proto:/var/lib/clickhouse/format_schemas/flow.proto
      - ./clickhouse/protocols.csv:/var/lib/clickhouse/user_files/protocols.csv
    depends_on:
      - kafka`

[lspgn]

Hello, Unfortunately I am not supporting confluentinc/cp-kafka:latest (the compose example uses bitnami). Can you try with the repo version? You will need the Kafka logs to see if there is an issue.

[KawtherGacem]

bitnami image is no longer available for free through Docker Hub. is there another kafka image that goflow2 supports?

[lspgn]

Hi @KawtherGacem, Would you be able to check out https://github.com/netsampler/goflow2/commit/14a9da938cf43010c80287d9f0777dedee7c55ba