vscode-kafka icon indicating copy to clipboard operation
vscode-kafka copied to clipboard

Enable query of topic

Open half-metal opened this issue 5 years ago • 1 comments

As of latest version, the topic is merely consumed, but does not allow querying or consuming the entire topic to see all results.

If extension were able to query by date range, then this would be excellent tool for troubleshooting any dev and production issues that may have already occurred in the past.

There is an alternative kSQL add-on, but I believe this depth of querying is typically not required by many users. Whereas this extension is straightforward and easy to get started.

half-metal avatar Jan 08 '20 22:01 half-metal

I like this idea, but there are a couple of things that needs to be done to enable it.

The kafka protocol allows fetching offsets by timestamp (ListOffset request), which could be used to do this, without having to scan all topics of a partition. My plan is to move to Kafja.js (away from kafka-node) which has this implemented (kafka-node does not). Kafka.js, on the other hand, is missing a few admin features.

I will try to contribute the missing features to Kafja.js, which should open up for this to be possible in the near future.

jlandersen avatar Jan 19 '20 19:01 jlandersen