circuitbreaker icon indicating copy to clipboard operation
circuitbreaker copied to clipboard

One single success request would cause state switch HalfOpen to Closed !

Open halegreen opened this issue 2 years ago • 0 comments

If the latency of circuit remote call is higher than 1s,then One single success request would cause state switch HalfOpen to Closed easily!

While the best solution is to detect Consecutive Success Request is reached the target, then switch switch HalfOpen to Closed 。

Here is the unit test you can produce this problem。

func TestThresholdBreakerResets(t *testing.T) {
	called := 0
	circuit := func() error {
		if called == 1 {
			called++
			return nil
		} else {
			called++
			return fmt.Errorf("mock error")
		}
	}
	remoteCall := 0
	c := clock.NewMock()
    
    //  Breaker 阈值设置为1,错误数大于1 则触发 Breaker Open
	cb := NewThresholdBreaker(1)
	cb.Clock = c
    
    // 第一次 call circuit
	err := cb.Call(circuit, 0)
	if err == nil {
		t.Fatal("Expected cb to return an error")
	}
    remoteCall++
	fmt.Printf("remoteCall:%d, err:%v, state:%d \n", remoteCall, err, cb.state())
    
    // 第一次 call 后,第二次 会返回 breaker open 错误
	err = cb.Call(circuit, 0)
	if err == nil {
		t.Fatal("Expected cb to return an error")
	}
	if err != ErrBreakerOpen {
		t.Fatal("Expected cb to return ErrBreakerOpen")
	}
    remoteCall++
	fmt.Printf("remoteCall:%d, err:%v, state:%d \n", remoteCall, err, cb.state())

    // 模拟等待已经超过指数退避时间,
    // 当第三次 call 后, 验证触发了 breaker 到 halfopen,然后再到 closed;
    // 第四次返回 circuit 的 mock error
    // 第五次返回 breaker open,此时又触发了 breaker open
	c.Add(cb.nextBackOff + 100000)
	for i := 0; i < 4; i++ {
		err = cb.Call(circuit, 0)
		remoteCall++
	    fmt.Printf("remoteCall:%d, err:%v, state:%d \n", remoteCall, err, cb.state())
	}
}

halegreen avatar Dec 26 '21 07:12 halegreen