elixir-omg
elixir-omg copied to clipboard
Watcher crash loop generates bursts of ethereum calls
Overview
When a watcher is misconfigured or a bug occurs, the watcher could enter a crash loop and restarts all the processes within. This in turn causes a burst of Ethereum calls to start all the processes. And because the crash is not self-recoverable, the burst could happen every few seconds (5 seconds?). This impacts integrators who relies on Ethereum node as a service e.g. Infura, and causes them to exhaust the quota rapidly.
Actual behaviour
When the watcher is misconfigured or a bug occurs, it brings the watcher into an irrecoverable crash loop and produces a constant burst of Ethereum calls every few seconds.
Expected behaviour
When the watcher is misconfigured or a bug occurs, the watcher should have an incremental backoff period for retries.
This is quite a concerning issues. Did this ever occur? Is there a way to replicate this? Have you seen it in the wild and can provide logs?