parseable icon indicating copy to clipboard operation
parseable copied to clipboard
Published 1 month ago • verified publisher icon parseablehq

Optimize Ingestion

Open parmesant opened this issue 1 year ago • 1 comments

During ingestion we accept a batch of events but still ingest them sequentially. Try to optimize that flow to improve ingestion.

parmesant avatar Sep 30 '24 10:09 parmesant

One naive way could be to spawn tokio tasks for each record but will the task overhead be greater than the resulting throughput improvement?

parmesant avatar Sep 30 '24 10:09 parmesant