Polly
Polly copied to clipboard
Circuit not closing when used in a nested way under concurrent situations
Summary:
We are using Polly circuit breaker policy to wrap our calls to Redis cache instace. As part of testing, we noticed that in concurrent situations, incase if the circuit breaker becomes open, it never closes even after the configured breaker duration. Upon close inspection, we found a usage in our code base where we had a method A which was calling method B wrapped in a circuit breaker policy & method B was internally calling redis client method again wrapped in the same circuit breaker policy. This was not intended and we removed it. Then onwards we found that circuit breaker was working as expected in the same test, where once the breaker becomes open, it transition to half open and closes as expect with the breaker duration eclipsed.
I couldn't see any documentation on whether this usage was valid or if this the expected behaviour incase of the such nested calling of the same policy? Can someone clarify?
Steps / Code to reproduce the problem:
- Create a polly circuit breaker policy
- Call your I/O logic wrapped in policy.Execute() method
- Call this wrapped method again nested inside the same policy.Execute() method
- Call the method in step 3 in concurrent situations
- Trigger a circuit breaking condition