Kitura
Intermittent test failures in Travis
The CircuitBreaker tests appear to have started intermittently failing in Travis. My suspicion is that this is test fragility rather than a real problem, but it ought to be investigated.
Example: https://api.travis-ci.org/v3/job/461370473/log.txt
Test Case '-[CircuitBreakerTests.CircuitBreakerTests testStateCycle]' started.
/Users/travis/build/IBM-Swift/CircuitBreaker/Tests/CircuitBreakerTests/CircuitBreakerTests.swift:617: error: -[CircuitBreakerTests.CircuitBreakerTests testStateCycle] : XCTAssertEqual failed: ("closed") is not equal to ("open") -
/Users/travis/build/IBM-Swift/CircuitBreaker/Tests/CircuitBreakerTests/CircuitBreakerTests.swift:620: error: -[CircuitBreakerTests.CircuitBreakerTests testStateCycle] : XCTAssertEqual failed: ("closed") is not equal to ("open") -
/Users/travis/build/IBM-Swift/CircuitBreaker/Tests/CircuitBreakerTests/CircuitBreakerTests.swift:625: error: -[CircuitBreakerTests.CircuitBreakerTests testStateCycle] : XCTAssertEqual failed: ("closed") is not equal to ("halfopen") -
Test Case '-[CircuitBreakerTests.CircuitBreakerTests testStateCycle]' failed (1.026 seconds).
I've pushed a branch to trigger a Travis build so we can see some examples: https://travis-ci.org/IBM-Swift/CircuitBreaker/builds/461718861
Another recent intermittent failure: https://travis-ci.org/IBM-Swift/CircuitBreaker/jobs/511393203
Test Case 'CircuitBreakerTests.testBulkheadFullQueue' started at 2019-03-26 09:42:25.301
/home/travis/build/IBM-Swift/CircuitBreaker/Tests/CircuitBreakerTests/CircuitBreakerTests.swift:580: error: CircuitBreakerTests.testBulkheadFullQueue : XCTAssertEqual failed: ("2") is not equal to ("3") -
Test Case 'CircuitBreakerTests.testBulkheadFullQueue' failed (2.006 seconds)
The circuit breaker tests are run relying on them being synchronously when they are async. This tries to be enforced using sleep but I think this doesn't always work causing the intermittent failures. I attempted to fix this in this in pr #52 however even when using dispatch groups I was getting test failures. I am unable to work in this further but will leave the PR up in case someone wants to look at in when they come back to this issue.
