Add support for timestamp offset

[atoulme]

trafficstars

Is your feature request related to a problem? Please describe.

When collecting data, we may need to offset timestamps because the source or destination operates in a different timezone. In particular with HEC, since we provide the timestamp as a millisecond unix epoch value, we lose the notion of timezone.

Describe the solution you'd like

We are looking to support adding a fixed timestamp offset to add or remove from the timestamp value on logs, metrics and traces.

This offset could be a HEC exporter/receiver specific setting or a processor, or part of an existing processor.

Describe alternatives you've considered

Obviously, it would be best if all systems were all configured with the same timezone, and that is the recommendation. If possible, it would be best to use UTC throughout.

Additional context

No response

[github-actions[bot]]

Pinging code owners: @atoulme @keitwb. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[evan-bradley]

@dmitryax Could you look into this?

[dmitryax]

It doesn't seem like something that should go to the HEC exporter. It might be generally useful mutation. @TylerHelmuth do you think this potentially can be achieved with OTTL in future?

[TylerHelmuth]

Yes, OTTL (TQL) supports setting the timestamp field today, although probably not intelligently (you have to supply a hard coded value essentially). But a more intelligent function could be added.

[github-actions[bot]]

Pinging code owners: @TylerHelmuth @kentquirk @bogdandrutu. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

If you could give me an example, even just pasted here on this issue, I can help add it to the transformprocessor. I'm not sure how the fixed hardcoded value is provided and in what unit - nanos, ms, or yet another duration.

[TylerHelmuth]

@atoulme at the moment the transform processor supports setting the time like:

transform:
  traces:
    queries:
      - set(start_time_unix_nano, 100000000) 
      - set(end_time_unix_nano, 200000000) 

[atoulme]

OK, this is to set the time. Can I increase or decrease the existing timestamp value?

[TylerHelmuth]

We don't have have the concept of addition or subtraction in the grammar yet, but it could be added.

[atoulme]

or subtracted (see what I did there)

[TylerHelmuth]

@atoulme math has been added to the OTTL. Does it solve this issue?

[atoulme]

It doesn't really as we need to change timestamps of all metrics, traces, and logs records, and across their sub-elements. In any case, I think this issue is no longer a problem for me as I have implemented my own processor - see #16562. Closing.