opentelemetry-python-contrib icon indicating copy to clipboard operation
opentelemetry-python-contrib copied to clipboard

Flaky test with pypy

Open xrmx opened this issue 2 years ago • 0 comments

Describe your environment

Got the a fastapi test fail on CI

Steps to reproduce

tox -f pypy3-fastapi -- -ra --benchmark-json=pypy3-fastapi-ubuntu-20.04-benchmark.json

What is the expected behavior?

Tests always pass

What is the actual behavior?

=================================== FAILURES ===================================
_____ TestAutoInstrumentationHooks.test_basic_post_request_metric_success ______

self = <tests.test_fastapi_instrumentation.TestAutoInstrumentationHooks testMethod=test_basic_post_request_metric_success>

    def test_basic_post_request_metric_success(self):
        start = default_timer()
        response = self._client.post(
            "/foobar",
            json={"foo": "bar"},
        )
        duration = max(round((default_timer() - start) * 1000), 0)
        response_size = int(response.headers.get("content-length"))
        request_size = int(response.request.headers.get("content-length"))
        metrics_list = self.memory_metrics_reader.get_metrics_data()
        for metric in (
            metrics_list.resource_metrics[0].scope_metrics[0].metrics
        ):
            for point in list(metric.data.data_points):
                if isinstance(point, HistogramDataPoint):
                    self.assertEqual(point.count, 1)
                    if metric.name == "http.server.duration":
>                       self.assertAlmostEqual(duration, point.sum, delta=30)
E                       AssertionError: 36 != 2 within 30 delta (34 difference)

Additional context

https://github.com/open-telemetry/opentelemetry-python-contrib/actions/runs/8647172961/job/23708138454#step:6:490

xrmx avatar Apr 11 '24 13:04 xrmx