OTEP: Extending attributes to support complex values

[lmolkova]

Related to #4468, #4414

This OTEP suggests a path forward on how to support complex attributes. It proposes to:

• allow complex attributes on all signals by default on the API and SDK level
• discourage them on metrics, resources, instrumentation scope, and identifying attributes on entities. If complex attribute is provided, it's probably by mistake, but we let backend decide.
• provide guidance in semconv to default to flat attributes whenever possible and develop conventions with an assumption that complex attributes are not indexed, not queryable and not efficient.
• documents how limits can apply to complex attributes (leaf nodes are counted towards the attribute count limit and are truncated based on the value limit) and suggests other ways to protect from users misusing complex attributes.

Having experimental phase for complex attributes limits the impact of this change and gives backends a graceful period to update.

[carlosalberto]

Is there a prototype for this?

[trask]

This OTEP doesn't include any technical detail about the reversal of a decision that was agonized over for months.

@dyladan can you add this as a code review comment so we can have a discussion about it? (I've found that mainline discussions quickly become entangled or lost on high traffic spec PRs) thanks

[github-actions[bot]]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[trask]

Is there a prototype for this?

https://github.com/open-telemetry/opentelemetry-python/pull/4587

[pellared]

OTel Go prototype: https://github.com/open-telemetry/opentelemetry-go/pull/6809

The idea was approved during today's SIG meeting.

We are in favor of extending the attributes to support complex values and we are blocking the stability of OTel Go Logs because of this.

[adriangb]

Would it be helpful to prototype this in the Python libraries?

[lmolkova]

@adriangb ptal at this one - https://github.com/open-telemetry/opentelemetry-python/pull/4587, would appreciate any feedback

[carlosalberto]

Minor comments but nothing blocking. Curious about the hashing part in further prototypes but LGTM

[trask]

@open-telemetry/android-maintainers @open-telemetry/cpp-maintainers @open-telemetry/dotnet-maintainers @open-telemetry/erlang-maintainers @open-telemetry/go-maintainers @open-telemetry/java-maintainers @open-telemetry/javascript-maintainers @open-telemetry/php-maintainers @open-telemetry/python-maintainers @open-telemetry/ruby-maintainers @open-telemetry/rust-maintainers @open-telemetry/swift-maintainers @open-telemetry/collector-maintainers @open-telemetry/collector-contrib-maintainers

please review if you have a chance, thanks!!

[dmitryax]

Should we require SDKs to provide clear guidance on when the complex attributes can be used and when they shouldn't be? Maybe as part of comments or documentation in particular places where the attributes can be set?

[trask]

Should we require SDKs to provide clear guidance on when the complex attributes can be used and when they shouldn't be? Maybe as part of comments or documentation in particular places where the attributes can be set?

hi @dmitryax! does this section of the OTEP address your concern?

API documentation and spec language around complex attributes SHOULD include language similar to this:

Simple attributes SHOULD be used whenever possible. Instrumentations SHOULD assume that backends do not index individual properties of complex attributes, that querying or aggregating on such properties is inefficient and complicated, and that reporting complex attributes carries higher performance overhead.