eliza icon indicating copy to clipboard operation
eliza copied to clipboard

fix: (core) Add circuit breaker pattern for database operations

Open augchan42 opened this issue 1 year ago • 3 comments

Implements circuit breaker pattern to handle database failures gracefully and prevent cascading failures. Fixes #712.

Changes:

  • Adds CircuitBreaker class with CLOSED, OPEN, and HALF-OPEN states
  • Introduces BaseCircuitBreakerAdapter for database adapters
  • Configurable failure thresholds and recovery timeouts
  • Automatic recovery attempts in HALF-OPEN state
  • Detailed logging of circuit breaker state changes

Circuit breaker configuration:

  • Opens after 5 consecutive failures (configurable)
  • Resets after 60 seconds in OPEN state
  • Requires 3 successful operations in HALF-OPEN state to close

This helps prevent overwhelming failed database connections and provides graceful degradation during outages.

Discord username

hosermage

augchan42 avatar Nov 30 '24 14:11 augchan42

Hi, I like the idea but it changes the overall use of DatabaseAdapter. Hence, I think might cause breaking changes. Is there anyway you could implement this into the DatabaseAdapter class itself?

shakkernerd avatar Nov 30 '24 23:11 shakkernerd

Ok, I changed the code to the way I think you wanted... please take a look.

On Sun, Dec 1, 2024 at 7:25 AM Shakker Nerd @.***> wrote:

Hi, this looks good to me but changes the overall use of DatabaseAdapter. Hence, I think might cause breaking changes. Is there anyway you could implement this into the DatabaseAdapter class itself?

— Reply to this email directly, view it on GitHub https://github.com/ai16z/eliza/pull/719#issuecomment-2509475568, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE32LKLHEHAGOC2QTVVNNX32DJCM7AVCNFSM6AAAAABSYR5EZ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBZGQ3TKNJWHA . You are receiving this because you authored the thread.Message ID: @.***>

augchan42 avatar Dec 01 '24 02:12 augchan42

Great, now its better. I will test locally and then merge in if no issue.

shakkernerd avatar Dec 01 '24 07:12 shakkernerd

lgtm, will leave for @shakkernerd to merge We're moving fast, so there are some small merge conflicts in package lock, whoever is maintaining can just fix and merge

lalalune avatar Dec 02 '24 00:12 lalalune