opentelemetry-specification icon indicating copy to clipboard operation
opentelemetry-specification copied to clipboard

Change map to key-value pair collection in Logs Data Model

Open pellared opened this issue 1 year ago • 1 comments

Why

Fixes https://github.com/open-telemetry/opentelemetry-specification/issues/3931

Performance

Checking for duplicates introduces performance overhead which can be undesirable for systems where logging should be high-performant. Changing the data model definition would allow postponing the deduplication to the exporters which require it (e.g. OTLP). Other exporters (e.g. console exporter) can use the key-values as provided.

Example benchmarks

  • https://github.com/open-telemetry/opentelemetry-rust/pull/1142
  • https://github.com/pellared/opentelemetry-go/pull/10

Better bridging of existing logging libraries

Some logging libraries allow duplicate keys. The change is needed for faithfully bridging existing logging systems.

For instance slog (the structured logging package from Go the standard library) allows it. See: https://go.dev/play/p/fzIswL0Le7j.

Consolidation

This is how C++, Rust, .NET already define map<string, any>. This is also how Go SIG wants to define it. The main reason is performance to avoid deduplication until necessary.

What

Change map<string, any> to []keyval<string, any> to allow having duplicated keys.

The current definition of map<string, any> is not using a normative language that would require to implementation to handle key-uniqueness. Therefore, I do not see this change as breaking.

Remarks

It is not planed to change anything in OTLP proto definition. More: https://github.com/open-telemetry/opentelemetry-proto/pull/533#issuecomment-1994336347

pellared avatar Mar 13 '24 13:03 pellared

@open-telemetry/specs-logs-approvers PTAL

pellared avatar Mar 13 '24 13:03 pellared

Changing to draft per https://github.com/open-telemetry/opentelemetry-specification/pull/3938#discussion_r1526582541

I plan to close this PR in a few days.

pellared avatar Mar 15 '24 19:03 pellared

I plan to close this PR in a few days.

@pellared Any reason we need to keep this open?

tigrannajaryan avatar Mar 21 '24 13:03 tigrannajaryan

FYI @open-telemetry/rust-approvers to inform our discussions in the future.

hdost avatar Mar 21 '24 22:03 hdost