data-prepper icon indicating copy to clipboard operation
data-prepper copied to clipboard

[BUG] Escaping of "/" in json pointers

Open Conklin-Spencer-bah opened this issue 1 year ago • 1 comments

Describe the bug

In my pipeline.yaml i am hoping to extract the value "foo" however in the field the key there are "/" slashes and it should be treated as one key. As seen below.

{ "kubernetes": { "labels": { "app.kubernetes.io/component": "foo" } } }

Data Prepper uses "/" for json pointers. So it would look like so: '/kubernetes/labels/app.kubernetes.io/component'

However this does not capture the value thus it is skipped. I think it is because the "/" in component needs to be escaped. How do you escape values in Data Prepper?

To Reproduce

  1. Publish an event like the one structured above.
  2. Attempt to parse out the event value "foo" and store it into another field.

Expected behavior I was expecting to be able to escape the "/" with a "/" or of the escape patterns specified in the documentation. However this did not work.

Environment (please complete the following information):

  • OS: MacOSX

Additional context AWS Managed OpenSearch Ingestion Service

Conklin-Spencer-bah avatar Oct 23 '24 18:10 Conklin-Spencer-bah

I was able to get this to work by escaping with ~1 as the /. However, we have a validation blocking ~ in the key. I will make a change to allow this character since it is a json pointer standard

processor:
    - copy_values:
        entries:
          - from_key: "/kubernetes/labels/app.kubernetes.io~1component"
            to_key: "copied_field"

graytaylor0 avatar Oct 25 '24 15:10 graytaylor0