add maxConcurrentCanaries flag to limit concurrent progressing canaries

[louishalbritter]

This adds a flag to limit concurrent progessing canaries to avoid high requests of resources at once.

The flag will not take effect if set to "0", which is default.

Closes #1069

[stefanprodan]

Please rebase with main and force push instead of merging, this PR should have a single commit.

[louishalbritter]

I cleaned to git history up again. Can you please merge or deny this pull request?

[aryan9600]

hey @louishalbritter, sorry this has been pending for a long time. i'll try to take a look this week. generally, we try to avoid making changes to the pkg/controller/scheduler.go. the code in there is the backbone of flagger and is very complicated, one small change can break the entire state machine. i'll need some time to review and test this, while trying to confirm that its not possible to implement this without making changes there.

[louishalbritter]

Thank you. I totally understand this. We're running this change since month in production and didn't have any problems with it. Let me know if I can support you somehow.