First-class exporter support for OTLP+JSON

[breedx-splk]

The OpenTelemetry specification for OTLP is stable. Part of this specification includes a description of the protobuf-to-json mapping, with a few explicit deviations called out. This specification also indicates that telemetry data sent via HTTP POST may be encoded as either protobuf or json.

At present, there is no simple, out-of-the-box way for users to specify that they want to use JSON encoding for HTTP. There are constants to request an OTLP exporter from autoconfigure, and there are constants for specifying grpc vs. http/protobuf, but http/json is notably missing.

I have personally seen users asking for this capability. I think that there are two primary use cases for this:

1. Human-readable payloads that can be easily inspected for troubleshooting and/or manual analysis. For example, users might perform a packet capture using wireshark or leverage an intercepting http proxy. Although protobuf mappings can be created, it's often much faster and simpler for developers to simply look at human-readable json.
2. Ingest standardization with other platforms. Some backends (read: RUM) might find it simpler and quicker to support a single JSON encoding. When considering the current state of protobuf on the web (read: unwieldy!), it's likely that web applications will want to send client telemetry encoded as json instead of protobufs. Without JSON support from java exporters, Android clients are forced to send protobuf, which forces the backend to also support it.

So, opening this issue so that we can discuss and clear the way for JSON encoding from our OTLP exporter.

[jack-berg]

Some related issues on the topic: #5598, #3651, #3486.

Summarizing the reasons to not add:

• Use case for them is small, and its more code to maintain
• http/protobuf will produce smaller encoded payloads
• Requires inclusion of jackson, increasing binary size
• Gives more options for users to choose from without a clear benefit

Some backends (read: RUM) might find it simpler and quicker to support a single JSON encoding.

Do you have any known examples of this, or is it hypothetical? My read is that a server is always going to prefer the protobuf binary encoding because the smaller payloads require less resources to deserialize. And given that the deserialization is abstracted away to a library, the server's developer shouldn't care whether the payload is json or protobuf binary.

[daniel-redly]

Do you have any known examples of this, or is it hypothetical?

If exporting metrics to NewRelic via their metric API, the payload must be in JSON, see here

[breedx-splk]

Do you have any known examples of this, or is it hypothetical?

If exporting metrics to NewRelic via their metric API, the payload must be in JSON, see here

Thanks for the link, but I'm pretty sure that's a vendor-specific format and not OTLP right?

[daniel-redly]

Sorry, I realised after further research that NewRelic has a separate OTLP endpoint that they expect you to use