logstash-input-s3-sns-sqs
logstash-input-s3-sns-sqs copied to clipboard
Cannot process empty .gz files
We're using AWS ALB access logging to S3. Sometimes ALB uploads empty (0b) files with .gz extension and S3 notifies about them to SQS. As result, this input plugin tries to process this files as gzipped, but always fails with Incomplete message in read_file. We´ll throw skip_delete.
and start processing after visibility_timeout reached. This loop only ends on reaching message retention period in SQS (which is 4 days by default). I'll address PR to fix this soon.
Thank you very much, I’ll test & release next week. As a quickfix you should configure a DLQ, to filter these messages fast.
see: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-configure-dead-letter-queue.html
regards Christian
Has this problem been fixed?
As you can see, PR is still not merged. Now you can manually build plugin from my repo and install it.