phlare
phlare copied to clipboard
Storage: Flushing row group of profiles.
- Ingest straight into parquet buffer reduce memory footprint by using parquet buffer compression.
- Then once the buffer is full flush a chunk to either a row group or just file ???
- Chunk meta from the index.
- Finally every x hour or if the symbols+index is too big flush the block.
In a way we basically need to compact the profiles.parquet into a bigger file.