Confusing purpose for logging API

[bogdandrutu]

What are you trying to achieve?

Clarity about when logging API should be used.

What did you expect to see?

Confusion about this since first statement is contradicted by the second, so it is very unclear if we expect an application written in golang for example to use this API or for example "log/slog"?

Additional context.

Add any other context about the problem here. If you followed an existing documentation, please share the link to it.

The Logs API is provided for logging library authors to build log appenders, which use this API to bridge between existing logging libraries and the OpenTelemetry log data model.

The Logs API can also be directly called by instrumentation libraries as well as instrumented libraries or applications.

_{Tip: React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[pellared]

We expect that both approaches can be used, depending on the application’s needs. Using Go as an example:

1. Some applications may prefer using slog together with the otelslog bridge, if they are already using slog.
2. Some applications may prefer using the Logs API directly to emit log records and events following semantic conventions. This is especially useful when they can reuse attributes from other signals, or from attribute factories (e.g., generated by weaver). (For OTel Go we are not there yet due to opentelemetry-go#7034).
3. Some applications may prefer the Logs API directly to reduce performance overhead introduced by conversions in the bridge.
4. Instrumentation libraries should use the Logs API, to avoid coupling the library to a specific logging framework or bridge.

@bogdandrutu, it seems this clarification is what’s missing in the specification, correct? I think https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/README.md would be the best place for it.

References:

• OTEP 265 – Event Vision: Interoperability with Other Logging Libraries
• https://github.com/open-telemetry/opentelemetry-specification/pull/4352
• https://github.com/open-telemetry/opentelemetry-specification/pull/4382

[bogdandrutu]

But the "Logs API" is not a proper logging API, and recommending using it in lots of places (2-4) will actually suffer for more performance hit (by not having levels, having to always create objects in Java), being hard to use compare to a log.Debug"foo", and multiple other reasons.

So if we really want to have it as an API we should extend to be a proper API, otherwise we should keep the focus only on the library author case.

[bogdandrutu]

@pellared also, just reading the introduction https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/README.md#introduction I read that we are not exposing a "final user" API, but we integrate with other APIs.

[pellared]

But the "Logs API" is not a proper logging API, and recommending using it in lots of places (2-4) will actually suffer for more performance hit (by not having levels, having to always create objects in Java), being hard to use compare to a log.Debug"foo", and multiple other reasons.

I think the performance hit is a Java-specific thing and it does not come from the specification. We have severity levels, Enabled API and Processors in the SDK; that are there for performance in mind.

So if we really want to have it as an API we should extend to be a proper API, otherwise we should keep the focus only on the library author case.

I think languages can look into building a proper "frontend" for Logs API. I am just not sure whether it should not be something language-specific.

[trask]

@bogdandrutu is this what you're looking for? https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/api.md#enabled

[sfc-gh-bdrutu]

@bogdandrutu is this what you're looking for? https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/api.md#enabled

I am looking to clarify the purpose of the API, this API was designed to not be a user-facing API, then somehow we relaxed that but we never improved the API to be user facing, so we are in a confusing state where the API is "not that recommended" (see first statement) but "kind of recommended" (see the second statement) for final user consumption, but in my opinion it is not ready for consumption yet as designed.

More details

Please read these 2 sentences:

The Logs API is provided for logging library authors to build log appenders, which use this API to bridge between existing logging libraries and the OpenTelemetry log data model.

From this do you not understand that it should not be used by final users?

The Logs API can also be directly called by instrumentation libraries as well as instrumented libraries or applications.

From this do you not understand that it should be used by final users, but is not very convincing?

For final user consumption:

1. I do expect simple ways to record a simple log message (just body);
2. Simple to use APIs (right now even to check if something is "Enabled" via https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/api.md#enabled need to write 100+ chars).
3. Time being controlled by user is a problem because may not guarantee monotonicity;
4. I would like to have helpers around levels (classic logging APIs), etc.

In my opinion this is a half backed API if we intend to have it for final users consumption.

[pellared]

From https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/0265-event-vision.md#open-questions

What kind of ergonomic improvements make sense now that the OpenTelemetry Logs API is user-facing? (Keeping in mind the bundle size constraints of browsers and possibly other client environments.)

@bogdandrutu, do you think improvements are also needed for instrumentation library purposes? For example, in Go we could imagine a "logging frontend" (similar to slog

For example, in Go I imagine we could create a "logging frontend" (similar to slog design, see: https://go.dev/blog/slog#the-design-process). That said, I haven’t proposed anything in this direction yet since I don’t see strong demand at the moment.

For OTel Java, however, a more ergonomic "frontend Logging API" might be very useful. I could draft a spec PR along with a Go prototype. Do you think this is a reasonable direction, or would you suggest an alternative approach?

Side note: Thanks a lot for your feedback and comments. I really appreciate it! It is hard to express gratitude via GitHub 😉

[lmolkova]

I think it would be useful to document that OTel user-facing Logs API is primarily intended for reporting structured logs and events. It does not (and probably never will), provide convenience features for string formatting, which are the focus of traditional logging facades. Applications that have to stick to unstructured or semi-structured logs SHOULD use a third-party logging facade where it's still possible to use OTel API inside logging bridge to covert traditional logs to OTel.

Agree we should provide more convenience around enabled/ordering/levels over time intended for fully structured logs/events.

[pellared]

I’ve built a working prototype of what I call the OpenTelemetry Logging Facade as a candidate “OTel user-facing Logging API”.

• Repository: https://github.com/pellared/olog
• Documentation: https://pkg.go.dev/github.com/pellared/olog

I’d love feedback on:

1. Whether this design is viable for inclusion in a spec PR (or a spike / draft in the spec repo).
2. Which parts seem most problematic or unclear.
3. Whether naming (“Logging Facade”) is acceptable (or too confusing / conflicting).

Happy to submit a first draft PR or detailed write-up if people think this is promising.

[pellared]

Short summary after the Spec SIG meeting: Languages are free to create more “ergonomic OTel logging libraries” as facades over the Logs API, potentially hosted in Contrib. Their design will likely be language-specific. This issue doesn’t appear to be a high priority at the moment.

[pellared]

@sfc-gh-bdrutu, @bogdandrutu: kind ping 😉

[pellared]

Notes after SIG meeting:

This is needed for writing instrumentation libraries even now. It is also nice for anyone creating custom events semantic conventions and use weaver for code generation.

Languages should be free create user-facing, more convenient logging API. We can let languages evolve on top of the existing Logs (Bridge) API. Here is a prototype I created for OTel Go: https://github.com/pellared/olog. Providing an user-facing, more convenient logging has to be optional as some languages (e.g. .NET) may prefer using something provided by the ecosystem. Maybe for some languages using Logs API directly would also be good enough.