logstash-input-file
logstash-input-file copied to clipboard
Fixed issue of re-reading rotated files
trafficstars
This is a fix for https://github.com/logstash-plugins/logstash-input-file/issues/238
Premis
- logstash checks the file uniqueness using the inode number, so even if sincedb exists, it does not fully support changing the file name in all cases.
- In a cloud environment, app rebooting occurs frequently due to deployment using docker, and as a result, tens of gigabytes or peta logs are read again.
Solution
- If the path of sincedb is changed and the inode is the same, it is recognized as a different file and the file is read again.
- However, even when file rotation occurs, it is always read again, which caused a problem. To solve the problem, look at the path of sincedb and start with this path and if the inode is not changed, it is regarded as a rotated file and the value of sincedb is maintained.
❌ Author of the following commits did not sign a Contributor Agreement: a439c2159668d09d3bc0338e05158049bdd14e7b
Please, read and sign the above mentioned agreement if you want to contribute to this project
❌ Author of the following commits did not sign a Contributor Agreement: 5040fab
Please, read and sign the above mentioned agreement if you want to contribute to this project
Done.