Add random offset capabilities to desynchronize parallel workers
Hi, this is a feature request plus code to implement it--I'd be happy to adjust the DSL or implementation if you had different ideas for how to do this. The idea is to implement the pattern described here where when several nodes are running the same code, we avoid having their crons all fire at the same time by starting each job with a random sleep.
My thought was to make the sleep distribution configurable on a per-list level, so that you can have some jobs be fuzzier than others (have a daily job have an hourly offset, an hourly job have a 2 minute offset). So the schedule.rb could look like
every(1.hour).and_about(2.minutes) do
...
end
every(:sunday, :at => '12pm').and_about(1.hour) do
end
every '0 0 27-31 * *', random_offset: <% server_offset_seconds %> do
end
Jobs in the first list would fire anywhere in the first four minutes of the hour, while jobs in the second list will fire anywhere between 12:00 and 2:00 PM, with any second being equally likely.
While it's not merged you can use:
every 1.day, at: (Time.zone.parse('10:00') + rand(0..600).seconds).localtime do
the randomness will only happy whenever the crontab is generated - is that what you're going for?
a better approach might be to specify a list of jobs that should never run together and then using some sort of simple wrapper script, execute them serially.