logstash-filter-aggregate icon indicating copy to clipboard operation
logstash-filter-aggregate copied to clipboard

Pipeline crash if timeout_timestamp_field is missing from event

Open TheVastyDeep opened this issue 2 years ago • 3 comments

logstash 8.8.1 installed on Linux via RPM and run on the command line. aggregate is bundled with the RPM. Bundled JVM used.

If the timeout_timestamp_field is missing from an event then the pipeline crashes here with the error.

[ERROR][logstash.javapipeline ][main] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"main", :error=>"(NoMethodError) undefined method `time' for nil:NilClass",

Reproducible with

input { generator { count => 1 lines => [ '1' ] } }
output { stdout { codec => rubydebug { metadata => false } } }
filter {
        aggregate {
            task_id => "%{message}"
            timeout_timestamp_field => "foo"
            timeout => 5
            push_map_as_event_on_timeout => true
        }
    }
}

TheVastyDeep avatar Jun 17 '23 20:06 TheVastyDeep

Well, this option requires that timestamp field exists on each event. Or else this option can’t do its job.

fbaligand avatar Jun 18 '23 07:06 fbaligand

Yes, that is obviously true, but it would be trivial to produce a clearer error message. and arguably a pipeline crash is not the optimal outcome.

TheVastyDeep avatar Jun 18 '23 13:06 TheVastyDeep

Yes, I agree with you.

fbaligand avatar Jun 19 '23 07:06 fbaligand