data-prepper
data-prepper copied to clipboard
Add option to shut down and fail data prepper when any sub-pipelines are invalid
Is your feature request related to a problem? Please describe. The default behavior of Data Prepper core is to only shut down when no sub-pipelines are valid. This can lead to a confusing user experience, as someone who wants all sub-pipelines to be running correctly for data prepper to start.
Describe the solution you'd like
An optional parameter in the data-prepper-config.yaml
to change this behavior to fail to start Data Prepper when any of the sub-pipelines are not configured properly. This would then throw an exception that stops Data Prepper when any sub-pipelines are misconfigured.
fail_on_any_sub_pipeline_failure: true // defaults to false
Describe alternatives you've considered (Optional) A clear and concise description of any alternative solutions or features you've considered.
Additional context Add any other context or screenshots about the feature request here.