starbase icon indicating copy to clipboard operation
starbase copied to clipboard

Support CRON-style repeat execution with polling periods/splay

Open erichs opened this issue 2 years ago • 0 comments

Starbase config.yaml integrations configuration should support an optional minMinutesBetweenExecutions parameter, as well as a global splayMins parameter.

  • if splayMins is set, Starbase should sleep for a random period, up to the specified value in minutes, before beginning execution. This helps with stampeding herd issues for large installations, similar to this issue.
  • if minMinutesBetweenExecutions is set, Starbase should consult the .integrations folder for the given integration. If it exists AND a previous execution time was recorded AND the specified value in minutes has not yet elapsed, Starbase should skip the integration run. When Starbase kicks off an integration, it should record the current time in a known location in the .integrations folder. This mechanism allows for a configurable polling period, allowing Starbase to execute via CRON more frequently while still enforcing dwell times between integration runs.

erichs avatar Dec 01 '22 01:12 erichs