quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Support data transformation in quickwit for `queue` sources

Open sunisdown opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. The data in kafka(or some sources else) has it's format, but maybe the format is not supported by Quickwit. before we consume the data with quickwit, we need to transform the data and fit it into quickwit format

Describe the solution you'd like We can add a transform layer in quickwit, and consume the predefined format and transform the data into Quickwit supported format

Describe alternatives you've considered We can also do the transform outside quickwit, use systems like Vector

sunisdown avatar Jul 08 '22 06:07 sunisdown