opentelemetry-collector-contrib

opentelemetry-collector-contrib copied to clipboard

Is your feature request related to a problem? Please describe. I want to able to rename Trace attribute keys with the transform processor.

Describe the solution you'd like

 transform/key_rename:
    traces:
      queries:
        - replace_all_patterns(attributes.keys, "^prefix_+", "prefix.")

Describe alternatives you've considered None as I know of no other alternatives.

Additional context Regex named capture groups don't allow for . characters in the capture group and I want my attribute keys to follow the Otel naming standard of using .s.

Attempted Implementation

attributes/extract:
    actions:
      - key: http.url
        pattern: ^(?P<prefix_http_protocol>.*):\/\/(?P<prefix_http_domain>.*)\/(?P<prefix_http_path>.*)
        action: extract
transform/key_rename:
    traces:
      queries:
        - replace_all_patterns(attributes.keys, "^prefix_+", "prefix.")

Code Change Required https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/transformprocessor/internal/traces/traces.go#L147

Instead of changing key names you should be able to use the "set" function to create a new attribute with the existing attribute and then use the "delete" function to remove the old attribute. I think you need 0.56.0 to access the delete function.

Oh but I see that you want to change potentially all the keys, not specific ones. Ya I don't think we support that yet.

Regex named capture groups don't allow for . characters in the capture group and I want my attribute keys to follow the Otel naming standard of using .s.

Are you referring to the attributes processor with this comment? This is why you can't do pattern: ^(?P<prefix.http.protocol>.*):\/\/(?P<prefix.http.domain>.*)\/(?P<prefix.http.path>.*) ?

If it is only those three you could do

transform/key_rename:
    traces:
      queries:
        - set(attributes["prefix.http.protocol"], attributes["prefix_http_protocol"])
        - set(attributes["prefix.http.domain"], attributes["prefix_http_domain"])
        - set(attributes["prefix.http.path"], attributes["prefix_http_path"])
        - delete_matching_keys(attributes, "^prefix_+")

But I agree that we cannot currently change keys dynamically. I'm not sure if we would support that via a new path accessor (attributes.keys) or a specific function.

Regex named capture groups don't allow for . characters in the capture group and I want my attribute keys to follow the Otel naming standard of using .s.

Are you referring to the attributes processor with this comment? This is why you can't do pattern: ^(?P<prefix.http.protocol>.*):\/\/(?P<prefix.http.domain>.*)\/(?P<prefix.http.path>.*) ?

Yes that's correct.

If it is only those three you could do

transform/key_rename:
    traces:
      queries:
        - set(attributes["prefix.http.protocol"], attributes["prefix_http_protocol"])
        - set(attributes["prefix.http.domain"], attributes["prefix_http_domain"])
        - set(attributes["prefix.http.path"], attributes["prefix_http_path"])
        - delete_matching_keys(attributes, "^prefix_+")

But I agree that we cannot currently change keys dynamically. I'm not sure if we would support that via a new path accessor (attributes.keys) or a specific function.

I would still like to see the functionality to dynamically change keys without having to call set on every attribute but what you suggested is perfect and seems to be working well for now :)

Final Working Solution

# 1. From the http.url attribute, using RegEx named capture groups, create three new attributes:
# 
# - temp_http_protocol
# - temp_http_domain
# - temp_http_path
# 
# 2. Set the name of the new attributes to the following:
# 
# - temp_http_protocol -> prefix.http_protcol
# - temp_http_domain -> prefix.http_hostname (using http.host and net.peer.name instead if they exist)
# - temp_http_path -> prefix.http_path
# 
# 3. Delete the temp attributes as we don't need them anymore
# 
# 4. For Span's that have a name like HTTP(S) {{ HTTP_METHOD }}, set the name to the following:
# 
# - {{ HTTP_METHOD }} {{ HTTP_HOSTNAME}}

  attributes/extract:
    actions:
      - key: http.url
        pattern: ^(?P<temp_http_protocol>.*):\/\/(?P<temp_http_domain>.*)\/(?P<temp_http_path>.*)
        action: extract
  transform/url:
    traces:
      queries:
        - set(attributes["prefix.http_protocol"], attributes["temp_http_protocol"]) where attributes["temp_http_protocol"] != nil
        - set(attributes["prefix.http_path"], attributes["temp_http_path"]) where attributes["temp_http_path"] != nil
        - set(attributes["prefix.hostname"], attributes["temp_http_domain"]) where attributes["temp_http_domain"] != nil
        - set(attributes["prefix.hostname"], attributes["net.peer.name"]) where attributes["net.peer.name"] != nil
        - set(attributes["prefix.hostname"], attributes["http.host"]) where attributes["http.host"] != nil
        # 0.56 Only
        # - delete_matching_keys(attributes, "^temp_+")

  attributes/cleanup:
    actions:
      - key: temp_http_protocol
        action: delete
      - key: temp_http_domain
        action: delete
      - key: temp_http_path
        action: delete
  span/rename:
    include:
      match_type: regexp
      span_names: ["^(HTTP(S)*?) ((GET)|(POST)|(PUT)|(DELETE)|(PATCH)|(UPDATE)|(OPTIONS)*?)$"]
    name:
      separator: " "
      from_attributes: ['http.method', 'prefix.hostname']

IIUC, for now, we should design a new function which could change the key's name that matches the regex expression dynamically, such as replace_all_key_patterns ? @TylerHelmuth

Yes a new function could do it. It would be very similar to replace_all_patterns.

Yes a new function could do it. It would be very similar to replace_all_patterns.

I'd love to try this, could you assign to me? @TylerHelmuth

@fatsheep9146 its yours. Forewarned, accepting the function will probably be depended on some of the PRs in #11751 being merged first.

@fatsheep9146 the issue is closed. You should add the new function to pkg/telemetryquerylanguage