hollywood
hollywood copied to clipboard
Retry message failures
This PR is a continuation of #181 . It introduces two new options WithRetries and WithMaxRetries and aims to retry message failures on actor panics.
The retry logic is as follows;
pid := e.SpawnFunc(func(c *Context) {}, "foo", WithRetries(2), WithMaxRetries(2), WithMaxRestarts(1))
- Each message that panics is retried up to two times per message
WithRetries(2). - On the second retry, an
ActorUnprocessableMessageEventis broadcast to the eventstream containing the message and the message within the buffer is dropped. - Following two consecutive message retries (4 retries), max retries are reached `WithMaxRetries(2).
- The actor is restarted to restore actor state and the messages left in buffer are resumed except those that were unprocessable.
- Retries are reset, but assume another two consecutive retries and restart. Max restarts are reached and the actor terminates and cleans up.
Using without retry;
pid := e.SpawnFunc(func(c *Context) {}, "foo", WithRetries(0), WithMaxRestarts(1))
- Each message that panics is dropped and actor is restarted as per original logic.
Included in this PR is a test that covers the full retry logic as well as amended tests.
All other tests pass.