http-request icon indicating copy to clipboard operation
http-request copied to clipboard

Fix flaky tests

Open dserfe opened this issue 1 year ago • 0 comments

This PR is to fix flaky tests caused by tests shared status.

Problem

If test com.github.kevinsawicki.http.HttpRequestTest#customConnectionFactory runs before other tests, such as com.github.kevinsawicki.http.HttpRequestTest#headWithEscapedMappedQueryParams, the latter one will fail:

[ERROR]   HttpRequestTest.headWithEscapedMappedQueryParams:2998 expected:<us er> but was:<null>

A full list of tests that can be polluted by the shared status include 28 tests:

com.github.kevinsawicki.http.HttpRequestTest.putWithEscapedMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.putWithEscapedVarargsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.putWithMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.putWithVarargsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.singleSslSocketFactory
com.github.kevinsawicki.http.HttpRequestTest.singleVerifier
com.github.kevinsawicki.http.HttpRequestTest.basicProxyAuthentication
com.github.kevinsawicki.http.HttpRequestTest.deleteWithEscapedMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.deleteWithEscapedVarargsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.deleteWithMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.deleteWithVarargsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.getUrlEncodedWithPercent
com.github.kevinsawicki.http.HttpRequestTest.getUrlEncodedWithSpace
com.github.kevinsawicki.http.HttpRequestTest.getUrlEncodedWithUnicode
com.github.kevinsawicki.http.HttpRequestTest.getWithEscapedMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.getWithEscapedVarargsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.getWithMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.headWithEscapedMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.headWithEscapedVarargsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.headWithMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.headWithVaragsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.postWithEscapedMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.postWithEscapedVarargsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.postWithNumericQueryParams
com.github.kevinsawicki.http.HttpRequestTest.verifierAccepts
com.github.kevinsawicki.http.HttpRequestTest.getWithVarargsQueryParams
com.github.kevinsawicki.http.HttpRequestTest.postWithMappedQueryParams
com.github.kevinsawicki.http.HttpRequestTest.postWithVaragsQueryParams

Root cause and fix

The test failure is caused by the shared status pollution from com.github.kevinsawicki.http.HttpRequestTest#customConnectionFactory, it sets a new ConnectionFactory but doesn't reset its status at the end of the test, thus leads to other test failures. To clear the pollution, we should reset the ConnectionFactory in teardown methods.

dserfe avatar Oct 25 '23 22:10 dserfe