"Best practice" for testing whether boolean field exists is broken

[TheVastyDeep]

PR 12899 modified the best-practice doc to add a way to test whether a boolean field exists by using mutate+add_field to add a static value to [@metadata] and then use mutate+copy to conditinally overwrite. Nice trick! Unfortunately it is broken. Filter decoration (add_field etc) happens after the filter function executes. So the copy is done before the add_field. It has to be split into two mutate filters.

[rneha725]

Hi @TheVastyDeep, I tried understanding the issue, are you suggesting the following:

filter {
  mutate {
    # we use a "temporal" field with a predefined arbitrary known value that
    # lives only in filtering stage.
    add_field => { "[@metadata][test_field_check]" => "a null value" }
  }

filter {
  mutate {
    # we copy the field of interest into that temporal field.
    # If the field doesn't exist, copy is not executed.
    copy => { "test_field" => "[@metadata][test_field_check]" }
  }

If yes, I can make the changes

[TheVastyDeep]

Yes, that is exactly the change required.

[TheVastyDeep]

@karenzone Any comments?