elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

Capture additional info into failure documents

Open nielsbauman opened this issue 9 months ago • 2 comments

When a document fails in the ingest service and gets redirected to a failure store, we're adding some more information to aid debugging. This PR adds the pipeline (the pipeline that failed to ingest the document), pipeline_trace (the trace of pipelines that were executed on the document), processor_tag, and processor_type to the failure documents.

To accommodate for said changes, we're also updating the mappings of the failure store -- which we can do easily as failure stores are currently still behind a feature flag.

nielsbauman avatar May 04 '24 06:05 nielsbauman

Pinging @elastic/es-data-management (Team:Data Management)

elasticsearchmachine avatar May 04 '24 06:05 elasticsearchmachine

would it be possible to have an IT test?

I've updated 190_failure_store_redirection.yml to test for the newly added fields. Thanks for the suggestion.

nielsbauman avatar May 08 '24 12:05 nielsbauman