opentelemetry-java icon indicating copy to clipboard operation
opentelemetry-java copied to clipboard

AttributeMap does not respect last-value-wins semantics for duplicate keys (SpanBuilder.setAttribute vs Attributes.builder)

Open Sabaev opened this issue 3 weeks ago • 2 comments

Describe the bug According to the OpenTelemetry specification:

  • Attribute keys are strings
    https://opentelemetry.io/docs/specs/otel/common/#attribute

    “The attribute key MUST be a non-null and non-empty string… Keys that differ in casing are treated as distinct keys.”

  • Setting an attribute with the same key SHOULD overwrite the existing value
    https://opentelemetry.io/docs/specs/otel/trace/api/#set-attributes

    “Setting an attribute with the same key as an existing attribute SHOULD overwrite the existing attribute’s value.”

However, in OpenTelemetry Java 1.56.0, last-value-wins semantics are not consistently applied.
Attributes.builder().put("key", ...) behaves correctly, but SpanBuilder.setAttribute("key", ...) produces different observable results when the same key name is reused with different types.

The core issue:
AttributeMap does not treat duplicate string keys consistently, violating spec last-value-wins semantics.

Steps to reproduce JUnit 5 test (Java):

import static org.junit.jupiter.api.Assertions.assertEquals;

import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.data.SpanData;
import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor;
import io.opentelemetry.sdk.testing.exporter.InMemorySpanExporter;
import java.util.ArrayList;
import java.util.List;
import org.junit.jupiter.api.Test;

class AttributeLastWinsTest {

  @Test
  void lastWinSemanticConvention() {
    InMemorySpanExporter exporter = InMemorySpanExporter.create();

    SdkTracerProvider tracerProvider =
        SdkTracerProvider.builder()
            .addSpanProcessor(SimpleSpanProcessor.create(exporter))
            .build();

    OpenTelemetrySdk telemetry =
        OpenTelemetrySdk.builder()
            .setTracerProvider(tracerProvider)
            .build();

    // Span 1: Attributes.builder() with same key, different types
    telemetry
        .getTracer("test")
        .spanBuilder("okSpan")
        .setAllAttributes(
            Attributes.builder()
                .put("key", "value")
                .put("key", false)
                .build())
        .startSpan()
        .end();

    // Span 2: SpanBuilder.setAttribute(...) with same key, different types
    telemetry
        .getTracer("test")
        .spanBuilder("test")
        .setAttribute("key", "value")
        .setAttribute("key", false)
        .startSpan()
        .end();

    List<SpanData> finished = exporter.getFinishedSpanItems();

    // Span 1: last-wins behavior (correct)
    assertEquals(
        List.of(false),
        new ArrayList<>(finished.get(0).getAttributes().asMap().values()));

    // Span 2: last-wins behavior is incorrect
    assertEquals(
        List.of(false),
        new ArrayList<>(finished.get(1).getAttributes().asMap().values()));
  }
}

What did you expect to see? For both spans:

Only one attribute with key "key" should appear.

The final value should be the last assigned value: false.

What did you see instead? Attributes.builder() → correct last-value-wins behavior.

SpanBuilder.setAttribute(String, ...) → incorrect last-value-wins behavior.

This contradicts the specification requirement that setting the same key SHOULD overwrite the existing value.

What version and what artifacts are you using? Artifacts: io.opentelemetry:opentelemetry-api io.opentelemetry:opentelemetry-sdk io.opentelemetry:opentelemetry-sdk-testing org.junit.jupiter:junit-jupiter

Version: 1.56.0

dependencies {
    testImplementation("io.opentelemetry:opentelemetry-api:1.56.0")
    testImplementation("io.opentelemetry:opentelemetry-sdk:1.56.0")
    testImplementation("io.opentelemetry:opentelemetry-sdk-testing:1.56.0")
    testImplementation("org.junit.jupiter:junit-jupiter:5.10.0")
}

Environment Compiler: openjdk 17.0.8.1 2023-08-24 OS: MacOs 15.6.1

Tip: React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Sabaev avatar Dec 05 '25 14:12 Sabaev

@Sabaev nice find! would you be interested in sending a PR for this?

trask avatar Dec 05 '25 14:12 trask

@trask Sure, I’ll give it a try 🙂

Sabaev avatar Dec 05 '25 15:12 Sabaev

I investigated the inconsistent last-value-wins behavior in OpenTelemetry Java. The root cause is that AttributesMap stores attributes in a HashMap<AttributeKey<?>, Object>, so equality is based on the full AttributeKey object (including type), not on the raw string key. Because of this, attributes with the same name but different types are treated as different keys, which violates the spec.

During benchmarking (4, 8, 16, 32, 64, 128 attributes), all naive approaches—linear scans, wrapper keys, or simple delegation—either reduce throughput or increase memory footprint significantly. A custom map that stores attributes by raw string key is the only approach that is both spec-correct and fast. It also avoids the problems created when AttributesMap was changed to extend HashMap, which introduced an unsafe API surface and made it easy to bypass attribute limits.


Future plans

  1. Create a real AttributesMap implementation that:

    • Stores entries keyed by raw string name.
    • Ensures one key per string, regardless of value type.
    • Implements strict last-value-wins semantics.
    • Does not expose unsafe HashMap APIs.
  2. Replace the current AttributesMap extends HashMap with this custom implementation.

  3. Add benchmarks and tests to confirm:

    • Correct last-value-wins behavior.
    • No duplicate string keys.
    • Better or equal performance to the existing implementation.
  4. Submit a PR with the new implementation and supporting benchmarks/tests.

I plan to work on this at the beginning of 2026 during my New Year vacation.

Sabaev avatar Dec 10 '25 11:12 Sabaev