[Bug] kafka_fdw foreign table returns zero rows: Feature not supported: Foreign Data

[ehkim12]

Apache Cloudberry version

PostgreSQL 14.4 (Cloudberry Database 1.0.0 build dev) on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-5), 64-bit compiled on Mar 24 2025 12:34:11 (with assert checkin g)

What happened

Description:

I have successfully created and configured the kafka_fdw extension and a foreign table to read from a Kafka topic. However, when querying the foreign table, it returns zero rows with the following log message:

NOTICE: "Feature not supported: Foreign Data"

This seems to indicate the foreign table was created correctly but data retrieval is blocked or unsupported by the executor.

Environment:

OS: Docker container

Kafka: 5.5.0 (also tried 7.4.0)

kafka_fdw: latest build from GitHub

Database: CloudberryDB / GreenplumDB

FDW server created with broker localhost:9092

Additional Context:

Kafka topic contrib_regress is successfully created. Using command-line tools, I can publish and consume messages. Seems like the FDW is skipping execution due to unsupported features.

Please advise on how to enable execution or whether a fix is available.

What you think should happen instead

I think cloudberry has to get data from foreign database system.

How to reproduce

Steps to Reproduce:

Installed and enabled kafka_fdw:

CREATE EXTENSION kafka_fdw;

Set up Kafka/Zookeeper using Docker:

version: '2' networks: test: services: zookeeper: image: confluentinc/cp-zookeeper:5.5.0 ports: [ "2181:2181" ] environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: confluentinc/cp-kafka:5.5.0 ports: [ "9092:9092" ] environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.222.51:9092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

Created server and foreign table:

CREATE SERVER kafka_server FOREIGN DATA WRAPPER kafka_fdw OPTIONS (brokers 'localhost:9092');

CREATE USER MAPPING FOR PUBLIC SERVER kafka_server; CREATE USER MAPPING FOR gpadmin SERVER kafka_server;

CREATE FOREIGN TABLE kafka_test ( part int OPTIONS (partition 'true'), offs bigint OPTIONS (offset 'true'), some_int int, some_text text, some_date date, some_time timestamp ) SERVER kafka_server OPTIONS (format 'csv', topic 'contrib_regress', batch_size '30', buffer_delay '100');

Ran query:

SELECT * FROM kafka_test LIMIT 1;

Expected Behavior: Should return rows from the Kafka topic contrib_regress.

Actual Behavior: Returns zero rows with notice:

NOTICE: "Feature not supported: Foreign Data"

Operating System

OS: Docker container Linux mdw 4.18.0-553.el8_10.x86_64 #1 SMP Fri May 24 13:05:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Anything else

No response

Are you willing to submit PR?

• [ ] Yes, I am willing to submit a PR!

Code of Conduct

• [x] I agree to follow this project's Code of Conduct.

[github-actions[bot]]

Hi, @ehkim12 welcome!🎊 Thanks for taking the time to point this out.🙌

[tuhaihe]

This issue is similar to https://github.com/cloudberry-contrib/kafka_fdw/issues/2.