opentelemetry-python icon indicating copy to clipboard operation
opentelemetry-python copied to clipboard

[metrics] dictionary changed size during iteration

Open sk- opened this issue 2 months ago • 1 comments

Describe your environment

OS: Debian Python version: 3.12.9 (main, Apr 8 2025, 01:41:13) [GCC 12.2.0] SDK version: 1.38.0 API version: 1.38.0

What happened?

We have gotten some errors on Sentry while exporting metrics because the dictionary changed size during iteration.

The error happens on file opentelemetry/sdk/metrics/_internal/metric_reader_storage.py in collect at line 149.

Steps to Reproduce

Unfortunately this happens about three times a day and we haven't found how to trigger it.

Image

Expected Result

Metrics should be exported successfully.

Actual Result

Sometimes the metrics fails with the error: RuntimeError: dictionary changed size during iteration

Additional context

Stack trace Image

Would you like to implement a fix?

None

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

sk- avatar Oct 24 '25 15:10 sk-

Thanks for the report, I can see how this would happen. We should probably just copy the whole dict or its keys for a stable iteration. We don't want to lock the whole dictionary

aabmass avatar Oct 30 '25 16:10 aabmass