dd-trace-go icon indicating copy to clipboard operation
dd-trace-go copied to clipboard

[Flaky test] profiler: TestExecutionTraceRandom flake

Open katiehockman opened this issue 1 year ago • 2 comments

https://github.com/DataDog/dd-trace-go/actions/runs/7617175486/job/20745637927

Seeing occasional flakes in CI for TestExecutionTraceRandom/rate-0.066667 in the profiler. An example failure:

=== Failed
=== FAIL: profiler TestExecutionTraceRandom/rate-0.066667 (2.16s)
    profiler_test.go:573: observed 15 traces, outside the desired bound of [1, 11]
    profiler_test.go:573: observed 12 traces, outside the desired bound of [1, 11]
    profiler_test.go:596: failed after retry
    --- FAIL: TestExecutionTraceRandom/rate-0.066667 (2.16s)

=== FAIL: profiler TestExecutionTraceRandom (4.42s)

cc @nsrip-dd @felixge

katiehockman avatar Jan 23 '24 20:01 katiehockman

Thanks! I'll dig into this. This is an inherently random test, and I hoped to make it quite unlikely to fail in the good case. But if it's failing too frequently, I'll see if I can make it better.

For reference, looking at our CI app I see that this has failed ~a single digit number of times out of ~520 runs since the test was introduced:

image

nsrip-dd avatar Jan 23 '24 20:01 nsrip-dd

This seems to be happening more often lately. I saw another of these flakes today: https://github.com/DataDog/dd-trace-go/actions/runs/7672345551/job/20912652647?pr=2521

katiehockman avatar Jan 29 '24 17:01 katiehockman