parseable icon indicating copy to clipboard operation
parseable copied to clipboard

Staging data ingestion and query optimizations

Open nitisht opened this issue 2 years ago • 0 comments

  • [ ] Defer the memtable.push flow outside of the write path. For example make the memtable push async / use a separate thread.
  • [ ] The disk.push should be made aysnc, so server doesn't wait for the event to be written always. Also discussed in https://github.com/parseablehq/parseable/issues/441
  • [ ] Concat of events (in the memtable) happens at 16384 events in memory. We need to either get rid of concat completely or move it out of ingest / write flow.

nitisht avatar Jul 13 '23 07:07 nitisht