splunk-otel-java icon indicating copy to clipboard operation
splunk-otel-java copied to clipboard

Address disabled "flaky test" in ProbabilisticSnapshotSelectorTest

Open breedx-splk opened this issue 9 months ago • 1 comments

See https://github.com/signalfx/splunk-otel-java/blob/main/profiler/src/test/java/com/splunk/opentelemetry/profiler/snapshot/ProbabilisticSnapshotSelectorTest.java#L95

based on this comment.

Testing things with random chance is always a challenge, but maybe a simple retry could help.

breedx-splk avatar Mar 04 '25 17:03 breedx-splk

Testing things with random chance is always a challenge, but maybe a simple retry could help.

I'd prefer deleting the dest over adding a retry. In my opinion the problem in that test is with math. It assumes that when enough iterations are performed then the results for most of the iterations will be in +/- 10% of the expected value which doesn't seem to hold. I'm not skilled enough in math to tell whether the problem is in the number of iterations or whether the bounds set in the test leave too high of a probability for failure.

@breedx-splk could you figure out a way to assign this to @tduncan

laurit avatar Mar 05 '25 08:03 laurit