cli icon indicating copy to clipboard operation
cli copied to clipboard

feat(ingest): support setting fields for csv without header row + csv batching

Open lukasmalkmus opened this issue 2 months ago • 0 comments

CSV should be batchable as well!

Also adds the CSV headers feature for CSV files without a header row.

In line with https://github.com/axiomhq/axiom-go/pull/305, I also addressed the batching behaviour here and made it less aggressive in time (flush batch every 5s instead of every 1s) but also introduced batch size based flushing (every 10k), whatever condition is met first. Also files are now batched instead of only stdin data.

lukasmalkmus avatar Apr 03 '24 16:04 lukasmalkmus