Prevent pyup bot from flooding CI/integrations (PR-storm/DDoS)

[webknjaz]

Hi,

So... I've noticed this a few month ago but got to creating this issue just now.

Story time (use-case) with suggested solutions

pyup creates branches right in repos (not forks). Then, it does push for each commit in those branches so in the end this results in 2 x number_of_commits builds in CI. (one build per push + one PR build for each of them)

This ends up hitting Travis' repo and org limits, which slows down and blocks almost all ongoing builds (because everything gets queued). And most of the other integrations have way stricter limits.

Also, we'd like to be good citizens and not waste free CI resources.

To reduce load on CI, I've started adding this if-clause to the repos in @aio-libs org like aio-libs/multidict@54912b7:

if: >  # Forbid running non-PR pushes from pyup bot
  not (type != pull_request AND branch =~ ^pyup\-scheduled\-update\-)

This cuts off half of the load but still is a bit hacky: having PRs from forks would do this automatically w/o need to copy-paste this snippet across all repos. Plus it only helps with Travis CI while others mostly don't have such "defense" mechanisms.

So the first request would be to make it possible having branches in other repo. I saw some integrations create fork on GitHub and use it for such purpose.

Now, each push event causes events. And pyup causes "push" on each commit. So integrations have to deal with lots of pushes (per-dependency to be precise). Auto-cancellation of builds in CI kinda helps a bit but (1) affects the whole repo including changes not influenced by pyup and (2) it's CI-specific (some integration don't have this feature or it works differently). This wouldn't be a problem if pyup created a series of commits and only then did one single push. It's the second request I want to make.

It might be less impactful for fast tiny test suites but if one run can take hours and is throttled by the integration it becomes disastrous!

Oh, and another multiplying factor is that pyup creates lots of PRs across the org nearly at the same time, which effectively creates a DDoS for our CIs.

I've raised the discussion with the @aio-libs crew earlier and @samuelcolvin suggested that pyup could do some cross-org schedule spreading across the week (request # 3). Think of it as a load balancer.

[samuelcolvin]

Makes sense, also surely it would be easier for pyup's server if you spread the load of creating PRs throughout the week/month?

[das7pad]

This issue seems to be related to the second request: https://github.com/pyupio/pyup/issues/280

[Jwomers]

Interesting and this makes sense, thanks for surfacing this. Cross-org spreading of new PRs could definitely be an easier way to solve this for now. Will look into this.

[belugame]

For our internal CI it is also too much if pyup creates a lot of merge requests at once. It would be great if one could add a optional random sleeping duration which pyup uses to spread out creating the MRs. Similar to unattended-upgrades in the debian world.

random_int=`shuf -i 0-1800 -n 1`
sleep $random_int