serverless-scaleway-functions icon indicating copy to clipboard operation
serverless-scaleway-functions copied to clipboard

Rate limit error when deploying a lot of functions with trigger

Open tdelmas opened this issue 2 years ago • 2 comments

When deploying ~20 functions with ~10 triggers each, serverless deploy hits rate limits

Deploying triggers...
Environment: linux, node 16.18.1, framework 3.25.1, plugin 6.2.2, SDK 4.3.2
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Error: AxiosError: Request failed with status code 429
    at manageError (/github/workspace/node_modules/serverless-scaleway-functions/shared/api/utils.js:20:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 3)
    at async Promise.all (index 1)

The code handling deployment:

https://github.com/scaleway/serverless-scaleway-functions/blob/master/deploy/lib/deployTriggers.js#L19-L23

could be improved in my opinion to avoid hitting rates limits:

  • It does all the requests in parallel (per function and per trigger), so x functions with y triggers will generate x*y parallel requests to the scaleway API.
  • It always deletes and recreate all triggers even if there are no changes.

tdelmas avatar Dec 22 '22 13:12 tdelmas

Thank you for analysis 👍

I'll try to improve the deploy trigger workflow, there is a complete delete/create of triggers each time because there are multiple sources of trigger CRUD (Scaleway Console, API, 3rd party tools etc...). The case an user wants to remove all of it's triggers even if created on Scaleway Console can be complicated.

thomas-tacquet avatar Dec 22 '22 13:12 thomas-tacquet

I'll try to improve the deploy trigger workflow, there is a complete delete/create of triggers each time because there are multiple sources of trigger CRUD (Scaleway Console, API, 3rd party tools etc...).

The current behaviors (after serverless deploy the list of trigger exactly match the list specify in the config) seams correct for me. (Only the implementation needs a little improvement in my opinion)

The case an user wants to remove all of it's triggers even if created on Scaleway Console can be complicated.

If the config file specifies an empty array, it should delete all existing trigger. If the config doesn't specify schedule I think it should do the same.

tdelmas avatar Dec 22 '22 13:12 tdelmas