data-prepper
data-prepper copied to clipboard
Add support for parse_json on complex json structures with arrays and nested arrays
Is your feature request related to a problem? Please describe. As a user with data in the format of
{
"array_one": [
{
"array_two": [
{
"field_one": {
"json_string": "{ \"key\": \"value\" }"
}
},
{
"field_one": {
"json_string": { \"key_two\": \"value-two\" }
}
}
]
},
{
"array_two": [
{
"field_one": {
"json_string": "{ \"key\": \"value\" }"
}
},
{
"field_one": {
"json_string": { \"key_two\": \"value-two\" }
}
}
]
}
]
}
I would like to parse the nested json for the json_string
in all objects of the nested arrays. This is not currently possible with the parse_json
processor as it only supports json_pointer
Describe the solution you'd like A new configuration to support this use case. One idea would be to utilize JSON path as the above example can be represented as this json path
- parse_json:
source_path: "array_one[*].array_two[*].field_one.json_string"
And this would convert all of the nested json fields to valid json format.
Describe alternatives you've considered (Optional) A clear and concise description of any alternative solutions or features you've considered.
Additional context Add any other context or screenshots about the feature request here.