log4stash icon indicating copy to clipboard operation
log4stash copied to clipboard

Not clear what these properties do

Open dariusdev opened this issue 4 years ago • 1 comments

Can someone explain what these properties do?

<BulkSize>2000</BulkSize> <BulkIdleTimeout>10000</BulkIdleTimeout> <IndexAsync>False</IndexAsync> <DropEventsOverBulkLimit>False</DropEventsOverBulkLimit>

dariusdev avatar Feb 26 '21 11:02 dariusdev

Sure,

Bulk properties

The indexing to Elasticsearch are sent by bulk and not one by one.
So on each call to "log" it will be accumulated until we have enough messages to sent or until a timeout is reached.

BulkSize

Number of messages to save in the memory before sending to Elasticsearch.

Possible values:

  • Any number that is greater than 1 - normal behavior as described
  • 1 - Don't accumulate and just send each message
  • 0 - Rely only on timeout. Accumulate all messages until timeout is reached regardless the current number of messages.

BulkIdleTimeout

As described, once this timeout (in milliseconds) is reached the current messages that are waiting in the bulk will be sent to elastic even if BulkSize isn't reached.

This action happens at a different thread than your code.

DropEventsOverBulkLimit

Sometimes users are ok with loosing some log messages in favor of memory and I/O. You can set this property to True in order to:

  • Send messages to Elastic only when timeout is reached
  • If a new message arrive and we already have BulkSize messages waiting - the message will be dropped and lost.
    A warning message will be written to the log4net ErrorHandler (LogLog)

IndexAsync

Whether to send the requests to Elasticsearch in an async manner.
Technically it will use .net async/await for this.

If sets to True each http request won't block your code and will call an async function that will send the request in different task.

If sets to False each http request will block your code which called the log function until it's done.

Pay attention that if BulkSize is greater than 1, not all calls to "log" will be blocked but only the last one that caused the bulk size to reach it's maximum size.

urielha avatar Mar 17 '21 10:03 urielha