Fix JSON filter @timestamp parsing to match JSON codec behavior

[groge]

Type of change

• bug: elastic/logstash#17931

Release notes

• Fixes an issue where JSON filter leaves @timestamp as a String. It now parses ISO8601 strings into a Timestamp, matching JSON codec behavior. On parse failure, adds _timestampparsefailure and preserves the original value in "_@timestamp".

What does this PR do?

• Adds special handling for @timestamp in Event.setField()
• Ensures @timestamp set via setField is converted to a Timestamp
• Aligns @timestamp handling between JSON codec and JSON filter
• Adds failure tagging and preserves the original value on invalid formats
• Fixes a bug where JSON filter would not parse @timestamp to a Timestamp object

Why is it important/What is the impact to the user?

• Guarantees consistent @timestamp handling across input (codec) and filter paths
• Simplifies pipelines that receive JSON with embedded @timestamp by removing the need for a separate date filter in common cases
• Improves visibility on parse failures (tag + original value preserved)

Implementation notes

• Detects FieldReference.TIMESTAMP_REFERENCE ("@timestamp") inside Event.setField() and uses the same initTimestamp() logic used by the constructor, then calls setTimestamp()
• On parse failure, adds _timestampparsefailure and stores the original value in "_@timestamp"
• Applies only to the root @timestamp; no impact to other fields or [@metadata]
• Negligible performance impact (one conditional + parse only when applicable)

Checklist

• [x] My code follows the style guidelines of this project
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
• [ ] I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• [x] Uses the same parsing path (initTimestamp) as the constructor
• [x] Verified tagging/original-value preservation on invalid formats
• [x] No impact to fields other than root @timestamp and no impact to [@metadata]
• [x] Backward compatibility maintained (root @timestamp coerced to Timestamp as intended)

How to test this PR locally

1. Pipeline (A) – JSON filter path

input { stdin {} }
filter { json { source => "message" } }
output { stdout { codec => rubydebug } }

Input:

{"@timestamp":"2023-12-01T10:30:00.000Z","foo":"bar"}

Expected:

• @timestamp: 2023-12-01T10:30:00.000Z (Timestamp object)
• foo: "bar"

2. Pipeline (B) – JSON codec path (for comparison)

input { stdin { codec => json } }
output { stdout { codec => rubydebug } }

With the same input, should produce the same result as Pipeline (A).

3. Failure case

{"@timestamp":"invalid-timestamp","foo":"bar"}

Expected:

• tags includes _timestampparsefailure
• "_@timestamp" contains "invalid-timestamp"
• @timestamp is set to the current time

Related issues

• Closes elastic/logstash#17931

Use cases

• Services/agents that emit JSON with @timestamp can rely on consistent application of the timestamp without a separate date filter
• Consistency between codec/json and filter/json in mixed environments

Screenshots

• N/A

Logs

• Success: @timestamp appears as a time object (e.g., 2023-12-01T10:30:00.000Z)
• Failure: _timestampparsefailure in tags, original value preserved in "_@timestamp"

[cla-checker-service[bot]]

💚 CLA has been signed

[github-actions[bot]]

:robot: GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify[bot]]

This pull request does not have a backport label. Could you fix it @groge? 🙏 To fixup this pull request, you need to add the backport labels for the needed branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• If no backport is necessary, please add the backport-skip label

[groge]

/backport 8.17 8.18 8.19 9.0 9.1

[groge]

run docs-build