firebase-tools

firebase-tools copied to clipboard

Description

There is an old issue we were facing with functions deployments reaching quotas: #3919.

From Firebase documentation:

When deploying large numbers of functions, you may exceed the standard quota and receive HTTP 429 or 500 error messages. To solve this, deploy functions in groups of 10 or fewer.

So why don't we just update this concurrency value from 40 to 10?

Scenarios Tested

Deployed 90 functions with:

firebase deploy --only functions

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

https://github.com/firebase/firebase-tools/issues/3919#issuecomment-1243786372:

I'm planning to review #4967 soon, but my feeling is that fighting the GCF quota limit isn't fool-proof way, especially since quota is different depending on your project's state, location, history, etc.

@taeold even if this isn't a perfect fix, could this be the base for at least it being user configurable or something maybe by an environmental variable? Then the default can be kept to the current value (40), but allows lower values if needed? I only suggest a environmental variable as I know command flags (which would probably make more sense for this) need to go through a whole review process. The currently function deployment is crippled with constant quota exceeded errors, so any fix (even if its temporary) has to be better than the current state.

Any news on this? Deploying to firebase is very time consuming because of these quotas. Would be great if we could just do firebase deploy --only functions and move on.

@longuniquename @rhodgkins @dikkeduif All great suggestions. We haven't done any deep investigations on what kind of value work best per function number, and the answer probably depends on the API quota allotted to the project.

I really like @rhodgkins's suggestion on exposing environment variable to configure this - that seems like the right first step. What if we do that and table this PR for later when we have some data that 10 is a better number (I'm not sure why our docs suggest 10 tbh)? If you have any data you can share with us, I'd deeply appreciate it.

Thanks for the update @taeold 🙂

If you have any data you can share with us, I'd deeply appreciate it.

If that's aimed at me I don't have anything I'm afraid. We've got a few environments/projects and there's no consistency on failed deployments due to quota errors. I could deploy locally without issue to one project, it could then fail deploying locally to a different project and then the Cloud Build might or might not succeed for either of the projects when I then try on there a while later 🤷‍♂️ all the projects have the same quota.

Also, as an aside, for all those projects the Write requests per minute quota is not editable anyway so we can't even increase the quota.

Regardless of the quotas I think firebase deploy should just auto retry to deploy. It's obvious we want to deploy our functions, no need to exit the process and tell us which ones failed to deploy. Is there a way to auto retry the ones that failed to deploy because of quota errors?

The way it works now is very inconsistent, sometimes it succeeds to deploy, sometimes it fails half of the functions and it's hard to understand exactly why.

Ideally it should be fire and forget. Life is too short to deal with quotas :)