Probably a new feature?

[3quanfeng]

Hi Guys, I have a quick question here, sorry I haven't read through the code... I am thinking if this tool can be easily extended to support other system than Elasticsearch? It could be something like the plugin model...for example, relay the events to other message queue such as rabbitmq or kafka queue(I know Kafka copycat can do that) so the end user of ElasticSearch can do customization on the events such as collecting only fields of the received event?

[ppine7]

Hi, @3quanfeng, I think this specific application was designed specifically for the purpose of fast batch indexing into ES, with ways to customize event transformation and indexing logic. Your idea is quite interesting, but would take it in a different direction - to become more of a generic ETL tool. There are quite a few commercial and open source tools that strive to do that - like LogStash, IBM's Informatica and many others, so not sure we would want to go in that direction just yet. :)

However, it is certainly a great idea to design this tool in such a way that others might easily plugin their own input/output logic if desired, and we should start thinking that way.

Thanks for sharing your ideas!

Marina