gax-java icon indicating copy to clipboard operation
gax-java copied to clipboard

BatcherImplTest.testThrottlingBlocking still flaky

Open chanseokoh opened this issue 3 years ago • 3 comments

The test is known to be flaky. So far, we did #1589 and #1600. (However, although it may just be me, I feel this became less flaky.)

Unfortunately, this failed again in #1613. The bright side of it is that we have a different error. Hopefully this reveals more information.

com.google.api.gax.batching.BatcherImplTest > testThrottlingBlocking FAILED
    org.mockito.exceptions.base.MockitoException: 
    No argument value was captured!
    You might have forgotten to use argument.capture() in verify()...
    ...or you used capture() in stubbing but stubbed method was not called.
    Be aware that it is recommended to use capture() only with verify()

    Examples of correct argument capturing:
        ArgumentCaptor<Person> argument = ArgumentCaptor.forClass(Person.class);
        verify(mock).doSomething(argument.capture());
        assertEquals("John", argument.getValue().getName());
        at com.google.api.gax.batching.BatcherImplTest.testThrottlingBlocking(BatcherImplTest.java:895)

chanseokoh avatar Jan 28 '22 20:01 chanseokoh

I've noticed it flaking again too. I'm at a loss here. I welcome another pair of eyes on this to try to fix it once and for all.

meltsufin avatar Jan 28 '22 20:01 meltsufin

Was the previous fix done by increasing timeout somethign from 10 ms to 50 ms? That could be just not enough difference. In many of our tests we use timeout as high as 20 sec, so maybe try just increasing it to something like 1sec. Or maybe this is the indicator of the logic itself having flaws. (btw, I've bee noticing flakiness myself too).

vam-google avatar Jan 31 '22 16:01 vam-google

It wasn't just the timeout. I thought the test was actually written is such a way that it wasn't testing what was intended. I thought I fixed it, but there still seems to be some kind indeterminism happening with the threads.

meltsufin avatar Jan 31 '22 17:01 meltsufin

Re-open if it's still flaky after #1775.

meltsufin avatar Oct 26 '22 15:10 meltsufin