clickhouse-scala-client
clickhouse-scala-client copied to clipboard
Clickhouse sink stream request to clickhouse instead of buffering in memory
Currently the ClickhouseSink uses groupWithin to buffer the items which need to be inserted into clickhouse. This causes a larger memory footprint and also bursts of data being sent to clickhouse.
We should basically rotate the stream to a different http sink whenever the batch size or batch timeout has been reached.