opentelemetry-python icon indicating copy to clipboard operation
opentelemetry-python copied to clipboard
verified publisher icon open-telemetry

Log stabilization current PRs overview

Open xrmx opened this issue 3 months ago • 7 comments

Plan

PR/issue impact alternatives notes
https://github.com/open-telemetry/opentelemetry-python/pull/4676 Removal of sdk LogData (no substitute) and LogRecord (now ReadableLogRecord and ReadWriteLogRecord). Change of typing in LogRecordProcessor.on_emit, in LogExporter.export. This will break openllmetry tests if they test against latest MERGED
https://github.com/open-telemetry/opentelemetry-python/pull/4654 deprecations of _events, typechecking issues for users MERGED
https://github.com/open-telemetry/opentelemetry-python/pull/4647 Renames a bunch of sdk and exporter Log related classes, some of the sdk classes are touched in #4676 . Most if not all the old classes names are kept around with a deprecation notice MERGED
https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3628 none MERGED, t, nlu
https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3626 none MERGED t, nlu
https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3625 none MERGED t, nlu
https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3624 none MERGED, t, nlu
https://github.com/open-telemetry/opentelemetry-python/issues/4319 this contains a suggestion to change the signature of Logger.emit to take separated params instead of a a LogRecord, this can break all the genai instrumentations we can do an implementation having both LogRecord and deprecate it later? e.g. #4737 MERGED
https://github.com/open-telemetry/opentelemetry-python/issues/4330 When moving the loggingHandler from sdk to separate instrumentation, people without the logging instrumentation installed will lose the functionality No PR yet
https://github.com/open-telemetry/opentelemetry-python-contrib/issues/3687 Removal of out of spec attributes from opentelemetry-instrumentation-logging No PR yet

nlu = introduce a new direct users for _logs api t = can be used as testbed for out of tree users

Feedback

issue impact notes fixed
https://github.com/open-telemetry/opentelemetry-python/issues/4836 warnings when using test utils from #4654 no
https://github.com/open-telemetry/opentelemetry-python/issues/4783 confusion handled by #4676, yes
https://github.com/open-telemetry/opentelemetry-python/issues/4838 breakage we can probably re-export LogData as deprecated inherited from ReadableLogRecord no

xrmx avatar Sep 12 '25 12:09 xrmx

I don't think any of the contrib issues are strictly needed for stabilization. We should address https://github.com/open-telemetry/opentelemetry-python/issues/4330 IMO.

FYI we have a label "log-breaking-changes" (issues) to track as well.

aabmass avatar Sep 12 '25 16:09 aabmass

I don't think any of the contrib issues are strictly needed for stabilization. We should address #4330 IMO.

Added the LoggingHandler move to the table. I've listed the contrib changes because they are users of some of the code we are touching and so helpful to reason about the impact.

FYI we have a label "log-breaking-changes" (issues) to track as well.

The other labeled issue is fixed by a PR listed in the table (#4647)

xrmx avatar Sep 15 '25 07:09 xrmx

So here's a try at serializing all these PRs:

  • [x] find some downstream users that may be interested in checking their stuff against OTel git. Do we have big downstream users of Events? openllmetry, we should ask them to test the events deprecation branch then :) Filed https://github.com/traceloop/openllmetry/issues/3451
  • [x] decide if we want to do the Logger.emit interface change #4319, if so start with a version that keeps backward compat and find a way to add the deprecations for the older interface
  • [x] merge a PR moving from events to logs for a single genai instrumentation and release it, see if any issues arise
  • [x] merge events api deprecation #4654 and evaluate how annoying it's for the other instrumentations, seek feedback from the downstream people, discuss and adapt accordingly - Dylan suggested to wait for this because users of the instrumentations don't have options. OpenLLMetry is fixed and released, we need to release ours instrumentation:
    • [x] release openai-v2
    • [x] update google-genai
  • [x] if first conversion from events to logs worked fine convert the others
  • [x] now think really hard on the effects of #4676, again downstream users can check if something breaks for them
  • [x] merge the PR doing the renames #4647 to double check that the old names are still around.
  • [x] adapt to the new names internally and seek for downstream user feedback. This a good time to check the documentation we have around (also in opentelemetry.io) and update that too
  • [ ] https://github.com/open-telemetry/opentelemetry-python-contrib/issues/3687 should be easy to do, sorry for your query if you were using them :)
  • [ ] Let's move the handler in #4330, this is a good chance to review the logging instrumentation code for other issues
  • [ ] tag a release breaking everything, YOLO
  • expect people complaining and issues reported :)

xrmx avatar Sep 15 '25 16:09 xrmx

Merging one of the gen AI PRs migrating off events SGTM if we want to play it super safe. I updated all those PRs to require v1.37 of the SDK/API which has your change attaching resource on emit, so they should be good to go..

We may want to wait on the events deprecation stuff for all the gen AI PRs migrating off events to go through, because people using those instrumentations will not be able to do anything about those warnings

DylanRussell avatar Sep 16 '25 21:09 DylanRussell

In opentelemetry.io we need to take care of the following references, that also means we don't have documentation for LogRecordLimits:

content/en/docs/languages/python/instrumentation.md:from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
content/en/docs/languages/python/instrumentation.md:processor = BatchLogRecordProcessor(ConsoleLogExporter())
content/pt/docs/languages/python/instrumentation.md:from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
content/pt/docs/languages/python/instrumentation.md:processor = BatchLogRecordProcessor(ConsoleLogExporter())

In opentelemetry-python-contrib, should be taken care of by https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3589:

instrumentation-genai/opentelemetry-instrumentation-google-genai/tests/common/otel_mocker.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-google-genai/tests/common/otel_mocker.py:        self._logs = InMemoryLogExporter()
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/conftest.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/conftest.py:    exporter = InMemoryLogExporter()
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/conftest.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/conftest.py:    exporter = InMemoryLogExporter()
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation/opentelemetry-instrumentation-botocore/tests/conftest.py:    InMemoryLogExporter,
instrumentation/opentelemetry-instrumentation-botocore/tests/conftest.py:    exporter = InMemoryLogExporter()

xrmx avatar Nov 20 '25 09:11 xrmx

In opentelemetry.io we need to take care of the following references, that also means we don't have documentation for LogRecordLimits:

content/en/docs/languages/python/instrumentation.md:from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
content/en/docs/languages/python/instrumentation.md:processor = BatchLogRecordProcessor(ConsoleLogExporter())
content/pt/docs/languages/python/instrumentation.md:from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
content/pt/docs/languages/python/instrumentation.md:processor = BatchLogRecordProcessor(ConsoleLogExporter())

@xrmx I can take care of opentelemetry.io for en/pt localization.

emdneto avatar Nov 20 '25 12:11 emdneto

1.39.0 contains all the changes to logs API and SDK (including events), 1.40.0 should contain the changes for the logging handler.

xrmx avatar Dec 03 '25 15:12 xrmx