Support the capture of complex attributes in span.

[Cirilla-zmh]

Background

According to semantic-conventions v1.38.0, generative AI instrumentations should emit chat messages as complex attributes both in span and in log, which depends on the option. However, when we try to migrate the semconv of OpenAI from v1.36.0 to v1.38.0 (see this PR), it turns out that we just support the capture of complex attributes in log now, and the feature for span is still on the way.

I believe this is a great user case which promotes the popularity of complex attributes and it's a great timing to do that. @trask and I have discussed this topic on the APAC Java SIG and then we have this issue.

Issues depend on this feature:

• https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/15174
• https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/12878

cc @anuraaga @lmolkova

Suggested Solution

We should add some test cases for extended span like this. Relevant infrastructures like marshalers should also be ready.

P.S. I guess we're able to set an object in Value<?> type as a span attribute yet but perhaps I'm wrong. I will add more information after studying it.

_{Tip: React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[trask]

hi @Cirilla-zmh, I gave this a try, but unfortunately it's a good deal more complex to implement this under incubation for Spans compared to for Logs because Spans are mutable during their lifespan (whereas Logs have no lifespan and are emitted directly on creation).

I don't think it's worth asking the maintainers to accept this complexity, given @jack-berg's earlier https://github.com/open-telemetry/opentelemetry-java/pull/7814#issuecomment-3544227460 and since it's only temporary complexity due to incubating features in the stable SDK (and not needed in the stable implementation), so no long-term benefit in thoroughly reviewing the complexity.

If I understand https://github.com/open-telemetry/opentelemetry-java/pull/7814, there's a user option to choose whether they want spans or events? would it be ok to only implement the event option initially? we are looking pretty good for being able to stabilize this in the Java SDK Feb release (https://github.com/open-telemetry/opentelemetry-specification/issues/4710#issuecomment-3603514568), at which time you'd be able to use it for spans too

[Cirilla-zmh]

If I understand #7814, there's a user option to choose whether they want spans or events? would it be ok to only implement the event option initially?

@trask Sure, I will try to serialize the attributes as a json string without the feature of complex attributes - we really have so many customers love this option. And if I couldn't find a reasonable solution, let's just emit logs before the Feb release. Thank you so much!

Let's leave this issue pending until we have the support.