eclipse-collections icon indicating copy to clipboard operation
eclipse-collections copied to clipboard

Fix or disable flaky test sumOfDoubleConsistentRounding().

Open motlin opened this issue 4 months ago • 1 comments

This test is flaky (and slow). After seeing it fail randomly a few times, I decided to record some evidence for anyone who wants to fix/reenable it. In a separate PR, I intend to disable the test.

I edited the test to use a fixed seed, and iterated until I found one that reliably causes the test to fail.

    @Test
    public void sumOfDoubleConsistentRounding()
    {
        DoubleFunction<Integer> roundingSensitiveElementFunction = i -> (i <= 99995) ? 1.0e-18d : 1.0d;

        Random random = new Random(1303L);
        MutableList<Integer> list = Interval.oneTo(100_000).toList().shuffleThis(random);
        double baseline = this.getExpectedWith(list.toArray(new Integer[]{}))
                .sumOfDouble(roundingSensitiveElementFunction);

        for (Integer batchSize : BATCH_SIZES)
        {
            this.batchSize = batchSize;

            ParallelIterable<Integer> testCollection = this.newWith(list.toArray(new Integer[]{}));
            assertEquals(
                    baseline,
                    testCollection.sumOfDouble(roundingSensitiveElementFunction),
                    1.0e-15d,
                    "Batch size: " + this.batchSize);
        }
    }

This gets all the Parallel versions of the test to fail with batch size 10000.

  • ImmutableListParallelListIterableTest.sumOfDoubleConsistentRounding()
  • MultiReaderFastListParallelListIterableTest.sumOfDoubleConsistentRounding()
  • ParallelCollectListIterableTest.sumOfDoubleConsistentRounding()
  • ParallelListIterableTest.sumOfDoubleConsistentRounding()
  • RandomAccessListAdapterParallelListIterableTest.sumOfDoubleConsistentRounding()
  • UnmodifiableMutableListParallelListIterableTest.sumOfDoubleConsistentRounding()

With error messages like:

org.opentest4j.AssertionFailedError: Batch size: 10000 ==> 
Expected :5.0000000000000995
Actual   :5.000000000000101

motlin avatar Aug 28 '25 15:08 motlin

Isn't a simpler solution to set the delta to 1e-14?

mohrezaei avatar Aug 29 '25 14:08 mohrezaei