kafdrop
kafdrop copied to clipboard
obsidiandynamics
Add Message Search Functionality
Overhaul search functionality prototype first introduced by @plmarcelo in PR #175. Improved performance by not reading the entire topic into memory and having a safety limit on the number of messages to read before the search aborts. Added a maximum results count. Added a date filter to allow user to specify which timestamp to start searching from. Added better output.
@nathanscottdaniels Can you please merge this feature, It really helps to quickly search.
@nathanscottdaniels Can you please merge this feature, It really helps to quickly search.
That's not how pull requests work. Only obsidian dynamics can merge it, at their discretion. If you want to use this feature, you can clone my fork and build it locally.
Hi Obsidian Dynamics - any thoughts on merging this PR? I'm looking for a Kafka tool that can do message searching within topics, and kafdrop with this PR looks ideal!
How search is implemented? Given that Kafka doesn't have any native feature to search (as far as I know), my concern is that this feature can be abused and cause some trouble in the Kafka servers. What do you think?
Anyway for sure conflicts must be resolved before we can start evaluating it.
How search is implemented? Given that Kafka doesn't have any native feature to search (as far as I know), my concern is that this feature can be abused and cause some trouble in the Kafka servers. What do you think?
Anyway for sure conflicts must be resolved before we can start evaluating it.
The code in this PR as well as its description answer all of your questions and concerns. But to summarize, searching is done within kafdrop by seeking to a point in time in the kafka topic and reading messages, filtering out those that don't match the search. I have put various safety measures in place to ensure the kafaka brokers are not abused, and this functionality is no more or less intensive the the brokers than a user of kafdrop flipping through pages of message previews.
I do not have a working Java development environment anymore so another volunteer will need to resolve the merge conflicts, sorry.
@nathanscottdaniels Can you allow me to push to the search branch in your fork?
@Bert-R I think you can just fork his branch and add more commits, then create another PR. This should maintain history and credits...
@nathanscottdaniels Can you allow me to push to the
searchbranch in your fork?
Just sent you an invite
@davideicardi It's compiling again and I've refactored it a bit for maintainability. One update is needed, based on a question I asked to Nathan. After that, I feel it's OK to merge this. The search behavior is just based on regular seeking and polling. I don't think it'll cause issues with the broker.