istio.io icon indicating copy to clipboard operation
istio.io copied to clipboard

TestCircuitBreaker is flaky

Open bison opened this issue 5 years ago • 3 comments

The TestCircuitBreaker test introduced in #6609 is flaky. It relies on examining the percentage of requests that succeed or fail to a service with circuit breaking configured, and ensuring the numbers fall within a given range. Unfortunately, the percentages tend to vary widely from run to run, so the test fails as often as it passes. The test was added at a time when this suite was not being run against pull requests regularly, so the failures went unnoticed for some time.

The test and corresponding documentation should be examined for any possible configuration changes that could make the results of the current method more consistent, or else the method of measurement in both the documentation and test should be changed.

Temporary fix: #7043

bison avatar Apr 14 '20 12:04 bison