opentelemetry-specification
opentelemetry-specification copied to clipboard
Include export timestamp to aid with inaccurate clocks
What are you trying to achieve?
For some data sources (e.g. browsers) the wall clock reading might be unreliable. There are several ways to deal with that (e.g. using NTP or extending the protocol to provide time sync) but those might be too heavy for the client library.
An alternative solution might be just registering the local time when the batch was exported by client and including it among attributes (e.g. telemetry.sdk.export_timestamp in Resource Attributes). The collector could support telemetry.sdk.receive_timestamp in a similar manner.
Having such information could be optionally used by the backend to make adjustments if the difference between the export_timestamp and timestamp when the message was acknowledge is above certain threshold.
Since that would be quite coarse-grained (there is a network-request in-between) I guess it cannot really be used to improve accuracy, but at least it could be used to catch really mis-configured clocks (wrong timezone, etc).
I think this may be better handled at the protocol level (e.g. new field in OTLP) instead of as a resource convention: reason being that this is quite specific information that is mostly useful if the backend actually does something with it automatically.
Since that would be quite coarse-grained (there is a network-request in-between) I guess it cannot really be used to improve accuracy, but at least it could be used to catch really mis-configured clocks (wrong timezone, etc).
Exactly, the grossly misconfigured clocks are the aim here. Doing it precisely would be much more significant effort.
I think this may be better handled at the protocol level (e.g. new field in OTLP) instead of as a resource convention: reason being that this is quite specific information that is mostly useful if the backend actually does something with it automatically.
I think the field could be optional since it would be useful only for sources that are known to have issues with clock sync (browsers are really the use-case I'm aiming at here). So maybe not deserving another field and 8 bytes of data in each record.
The way protobuf works is that fields are not sent over the wire if they are not set (or set to the default/zero). So for OTLP this would not be a problem.
Lightstep's old OpenTracing tracers include clock correction. The server would respond with its timestamps and the client would calculate an offset. It tends to cause a lot of trouble, but we also find that mobile and browser clocks are very often wildly off.