data-prepper icon indicating copy to clipboard operation
data-prepper copied to clipboard

Published 20 hours ago • verified publisher icon opensearch-project

Add Duration support for Data Prepper Configuration using PluginDurationDeserializer

Open asifsmohammed opened this issue 2 years ago • 1 comments

Is your feature request related to a problem? Please describe. Currently, only piplelines.yaml can be deserialized using PluginDurationDeserializer. It doesn't allow DataPrepperConfiguration to use Duration.

Describe the solution you'd like One solution I could think of is make PluginDurationDeserializer a generic class instead of just for plugins and make it public. We can configure the object mapper of DataPrepperAppConfiguration here with custom deserializer.

asifsmohammed avatar Jul 27 '22 20:07 asifsmohammed

Additionally, this can formalize the concept of a "Data Prepper duration". This is an input string that can be in the format of either an ISO-8601 string or expressed as "Ns" or "Nms" for a specified second or millisecond value.

dlvenable avatar Aug 15 '22 19:08 dlvenable

Fixed as part of https://github.com/opensearch-project/data-prepper/pull/1757

asifsmohammed avatar Sep 15 '22 19:09 asifsmohammed