serilog-sinks-seq icon indicating copy to clipboard operation
serilog-sinks-seq copied to clipboard

Durable sink should batch based on event size and count

Open jamie94bc opened this issue 9 years ago • 1 comments

We're logging lots of different sized events - some tiny, some huge. We ran into an issue with Seq refusing to accept a large batch of events which occurred because lots of large events were submitted in a single batch.

Rather than reducing the batch count, it would be great to have the the option to also batch by request size. This option would allow many smaller events to be sent at once (say 1000 per batch) and larger events to be sent in smaller batches.

Event with the changes in dev branch to drop large events, it's possible individual events may not exceed _eventBodyLimitBytes.Value but when batched will exceed's Seq's Raw Event Payload Limit.

By batching based on request size, the likelihood of events being dropped would be much lower.

jamie94bc avatar Feb 24 '16 09:02 jamie94bc

Thanks for the suggestion, definitely more opportunities to improve here :+1:

nblumhardt avatar Feb 25 '16 23:02 nblumhardt