opentelemetry-python icon indicating copy to clipboard operation
opentelemetry-python copied to clipboard

Add OTLP HTTP MetricExporter max_export_batch_size

Open tammy-baylis-swi opened this issue 7 months ago • 5 comments

Description

Adds support for HTTP OTLPMetricExporter configurable max_export_batch_size, like the gRPC OTLPMetricExporter already does (completed through issue https://github.com/open-telemetry/opentelemetry-python/issues/2710 with PR https://github.com/open-telemetry/opentelemetry-python/pull/2809).

This is currently much longer than the gRPC version because:

  1. HTTP protobuf representations of ResourceMetrics, ScopeMetrics, etc are not replace-able like the gRPC data classes
    • So references are stored and new protobuf objects are created immediately before yield/export
  2. protobuf does not define a DataPointT to encompass all metric types
    • So I've added some if-elif throughout for accessing data points, creating new metrics objects

Fixes https://github.com/open-telemetry/opentelemetry-python/issues/4577

Type of change

Please delete options that are not relevant.

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [X] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [X] This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • [X] Added unit tests
  • [X] Install OTLPMetricExporter locally and do the following:
  1. Set up 3 counters, with add(1) calls to each
  2. Init global MeterProvider using OTLPMetricExporter with max_export_batch_size=2 and endpoint as Collector (debug).
  3. Run to get export in 2 batches (2 counters + 1 counter) of 1 ResourceMetrics each.

Does This PR Require a Contrib Repo Change?

  • [ ] Yes. - Link to PR:
  • [X] No.

Checklist:

  • [X] Followed the style guidelines of this project
  • [X] Changelogs have been updated
  • [X] Unit tests have been added
  • [X] Documentation has been updated

tammy-baylis-swi avatar May 09 '25 18:05 tammy-baylis-swi

I think the aiohttp-client test failure is a hiccup from the recent release, not from changes in this PR.

tammy-baylis-swi avatar May 09 '25 19:05 tammy-baylis-swi

Working on some conflict resolution with changes introduced in https://github.com/open-telemetry/opentelemetry-python/pull/4564

tammy-baylis-swi avatar Jul 03 '25 17:07 tammy-baylis-swi

Hi @lzchen and @open-telemetry/python-approvers , please may I have a new review of this PR.

In 25e8be9 I resolved merge conflicts which required some redesign of HTTP metrics batching to match the timeout updates in https://github.com/open-telemetry/opentelemetry-python/pull/4564. That was the main change since last approval; the split helpers are the same.

The last Cassandra and Celery instrumentor test failures seem unrelated and are citing commit hashes like similar failures in other PRs.

tammy-baylis-swi avatar Jul 03 '25 21:07 tammy-baylis-swi

Hi @open-telemetry/python-approvers , any other revisions needed for this one?

tammy-baylis-swi avatar Aug 21 '25 18:08 tammy-baylis-swi

Hi again @open-telemetry/python-approvers , @open-telemetry/python-maintainers please could I have a review of this?

tammy-baylis-swi avatar Nov 12 '25 17:11 tammy-baylis-swi