Allow Heroku logs containing dyno metrics to be transformed into useable attributes

[jdeff]

Component(s)

OTTL

Is your feature request related to a problem? Please describe.

Heroku emits log lines containing metrics data for the various dyno instances running for an application. These log lines look similar to:

source=web.1 dyno=heroku.1234-5678-9012-3456 sample#memory_total=10917.88MB sample#memory_rss=10849.69MB sample#memory_cache=68.20MB sample#memory_swap=0.00MB sample#memory_pgpgin=3384pages sample#memory_pgpgout=82pages sample#memory_quota=14336.00MB

Using the syslog_receiver and key_value_parser you can get something almost useable. However, the values are still strings of human readable byte sizes (e.g. "68.20MB") and need to converted to an integer number of bytes in order for them to be useable.

Describe the solution you'd like

I would like to be able to parse these byte size values using the transformprocessor, which I am already using to shape other values, and convert them to an integer value of bytes. I propose adding a ParseBytes function to OTTL in the same vain as ParseJson and ParseCsv. This could be used with the following config:

processors:
    transform:
      log_statements:
        - context: log
          statements:
            - replace_all_patterns(attributes, "key", "sample#memory-total", "system.memory.usage.used")
            - set(attributes["system.memory.usage.used"], ParseBytes(attributes["system.memory.usage.used"])) where attributes["system.memory.usage.used"] != nil

Describe alternatives you've considered

I've looked at simply striping the units from these values, but the units are not guaranteed to be the same.

ParseBytes may be an ambiguous or confusing name (though it is what is used by go-humanize). Something like ParsePrettyByteSize or ParseHumanizedByteString might be clearer.

Additional context

No response