Implements safe evaluation of conditional expressions, logging the error without killing the pipeline

[andsel]

Release notes

Fix if statement expression evaluation against runtime errors that could crash the pipeline, just logging the offending event and continuing with the next in the batch.

What does this PR do?

Translates the org.jruby.exceptions.TypeError, IllegalArgumentException, org.jruby.exceptions.ArgumentError that could happen during EventCodition evaluation into a custom ConditionalEvaluationError which bubbles up on AST tree nodes. It's catched in the SplitDataset node. Updates the generation of the SplitDataset so that the execution of filterEvents method inside the compute body is try-catch guarded and defer the execution to an instance of AbstractPipelineExt.ConditionalEvaluationListener to handle such error. In this particular case the error management consist in just logging the offending Event.

Why is it important/What is the impact to the user?

This PR protects the if statements against expression evaluation errors, cancel the event under processing and log it. This avoids to crash the pipeline which encounter a runtime error during event condition evaluation, permitting to debug the root cause reporting the offending event and removing from the current processing batch.

Checklist

• [x] My code follows the style guidelines of this project
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• ~~[ ] I have made corresponding change to the default configuration files (and/or docker env variables)~~
• [x] I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• [x] add test case for #15091

How to test this PR locally

1. start with small pipeline like

input {
    stdin { codec => json_lines }
}
filter {
  if [path][to][value] > 100 {
    mutate { add_tag => "hit" }
  }
}
output {
  stdout { 
    codec => rubydebug
  }
}

and use the data samples from #16007 2. tinker with ifs also in the output, something like:

input {
    stdin { codec => json_lines }
}

filter {
  if [path][to][value] > 100 {
    mutate { add_tag => "hit" }
  } else {
    mutate { add_tag => "miss" }
  }
  mutate { add_tag => "after"}
}

output {
   if [path][to][value] > 100 {
        stdout {
            codec => rubydebug
        }
   }
}

3. then verify with batches containing ok and ko data. Add the following input for example:

file {
    path => "/tmp/pipeline_conditional_test_fixture.json"
    sincedb_path => "/tmp/logstash_andsel/sincedb"
    mode => "read"
    file_completed_action => "log"
    file_completed_log_path => "/tmp/processed.log"

    codec => json
}

Using a json file like:

{"path":{"to":{"value":101}}}
{"path":{"to":{"value":102}}}
{"path":{"to":{"value":"101"}}}
{"path":{"to":{"value":103}}}

4. whatever other test comes to your mind :-)

Related issues

• Relates #16007
• Closes #15091

Use cases

Screenshots

Logs