Doc Enhancement: copytruncate and Log Loss Risk

[hardikkhurana]

Fluent Bit documentation to explicitly warn users about potential log loss when using logrotate's copytruncate mode.

While we note that file rotation is handled, the docs don't address the risks inherent to copytruncate, specifically:

1. Race conditions: Logs can be lost in the brief window between copying and truncating the file.

2. Fluent Bit backpressure: If FLB is under backpressure, logs might be dropped if copytruncate occurs before they can be processed and sent.

Referencing the logrotate man page: "Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost." This crucial detail should be included in our documentation.

Even FileBeat's Documentation has a page on it - https://www.elastic.co/docs/reference/beats/filebeat/file-log-rotation

[eschabell]

@hardikkhurana do you mean the docs page for tail (see bottom) as the entry point for a warning?

[hardikkhurana]

@eschabell Yes, that sounds like a good place for it! The tail plugin's documentation is where users configuring log ingestion would typically look, making it an ideal spot to highlight the copytruncate risks.

"File rotation is properly handled, including logrotate's copytruncate mode." -> seems misleading.