qryn icon indicating copy to clipboard operation
qryn copied to clipboard

Published 20 hours ago • verified publisher icon metrico

BULK_MAXAGE / BULK_MAXSIZE - insert limits!?

Open bzed opened this issue 10 months ago • 2 comments

Hi,

according to the documentation there is are BULK_MAXSIZE BULK_MAXCACHE settings, but I can't find them in the code.

Is there any undocumented way I didn't find to limit the size of the inserts?

Thanks,

Bernd

bzed avatar Apr 23 '24 11:04 bzed

Now it sends as many as you can ingest during 100ms. Also we have a limit for ingestion of 50MB/sec before a request proceeds to be parsed.

akvlad avatar Apr 23 '24 11:04 akvlad

@bzed can you suggest the best solution you suggest about the BULK_MAXSIZE configuration? What should happen if the request overflows the configured value?

akvlad avatar Apr 25 '24 11:04 akvlad

Bulk size configuration

since 3.2.23

Two env vars were introduced:

  • BULK_MAX_SIZE_BYTES - max size of the bulk in bytes (default - infinite)
  • BULK_MAX_AGE_MS - max age of the bulk in milliseconds (default - 100), can't be less than 100

akvlad avatar Jun 13 '24 14:06 akvlad

@bzed this was implemented. Let us know if you have any feedback and/or comments.

Cloding as completed!

lmangani avatar Jun 20 '24 10:06 lmangani