firebase-tools
firebase-tools copied to clipboard
firebase
Can't deploy all functions consistently. Fails most of the time.
Hello;
I have a project with 169 cloud functions and need to set up automatic deployment to firebase in GitLab, but the deployment is always failed.
I tried with various node versions, also with various access tokens, but always got the same error. Locally one of the developers is able to deploy without any issue. But locally in my environment, it fails. Even when replicating the versions of node and firebase.
here is my .yml file ` image: node:14.16.1 before_script:
- cd functions
- npm I
script: - npx firebase use way-backend - npx firebase deploy --only functions --token $FIREBASE_TOKEN only: refs: - master changes: - functions/**/* `
first warnings like this with many functions
⚠ functions: got "Quota Exceeded" error while trying to update projects/xproject/locations/us-central1/functions/xxxxx. Waiting to retry...
and then got this one
Functions deploy had errors with the following functions: function1(us-central1) function2(us-central1) .... etc Because there were errors creating or updating functions, the following functions were not deleted To delete these, use firebase functions:delete
and not all functions throw the error only a few functions
and finally got this one
i functions: cleaning up build files... Cleaning up project directory and file based variables 00:01 ERROR: Job failed: exit code 1
Thanks in advance
Hey we're aware of issues when deploying over 100 functions, it's best to deploy the functions you changed specifically instead of the whole lot when you're at this scale.
Hey we're aware of issues when deploying over 100 functions, it's best to deploy the functions you changed specifically instead of the whole lot when you're at this scale.
usually, we deploy only changed functions, but we need to find a way to set up CI/CD on this project, also when we deploy partially the config changes are still undeployed!
@abeisgoat I'm also facing a quota issue, although I'm trying to deploy just 20 functions. Can you confirm if it's related or it's another issue? The output that I have in Functions dashboard is:
{"@type":"type.googleapis.com/google.cloud.audit.AuditLog","status":{"code":8,"message":"Build failed: Quota exceeded for quota metric 'Build Create requests' and limit 'Build Create requests per minute' of service 'cloudbuild.googleapis.com' for consumer 'project_number:<NUMBER>'."},"authenticationInfo":{"principalEmail":"<EMAIL>"},"serviceName":"cloudfunctions.googleapis.com","methodName":"google.cloud.functions.v1.CloudFunctionsService.UpdateFunction","resourceName":"projects/ticker-dev-4d59e/locations/southamerica-east1/functions/<FUNCTION_NAME>"}
Deploying the functions individually works as expected, but trying to deploy all of them at the same time outputs me the error above.
I have the same issue.
Error: Functions did not deploy properly.
Having trouble? Try firebase [command] --help
mac@bogon functions % firebase deploy --only "functions:sendMessageNotification,functions:unsubscribeUserFromBroadcast,functions:setStatusCount,functions:resubscribeUserToBroadcasts,functions:addUserToGroup,functions:deviceIdChanged,functions:sendMessageToBroadcast,functions:participantRemoved,functions:groupInfoChanged,functions:participantAdded,functions:getVirgilJwt,functions:sendUnDeliveredNotifications,functions:unsubscribeUserFromTopicOnDelete,functions:saveUidOnLogin,functions:sendNewCallNotification,functions:deleteMessageForGroup,functions:indexNewCall,functions:groupAdminChanged,functions:deleteMessageForBroadcast,functions:indexNewGroupCall,functions:groupEvents,functions:getTime,functions:sendMessagesForGroups,functions:deleteMessage,functions:indexPKToken,functions:subscribeToBroadcast"
=== Deploying to 'ichat-daac0'...
i deploying functions
i functions: ensuring required API cloudfunctions.googleapis.com is enabled...
i functions: ensuring required API cloudbuild.googleapis.com is enabled...
✔ functions: required API cloudfunctions.googleapis.com is enabled
✔ functions: required API cloudbuild.googleapis.com is enabled
i functions: preparing functions directory for uploading...
i functions: packaged functions (73.82 KB) for uploading
✔ functions: functions folder uploaded successfully
i functions: updating Node.js 10 function sendMessageNotification(us-central1)...
i functions: updating Node.js 10 function sendMessagesForGroups(us-central1)...
i functions: updating Node.js 10 function participantAdded(us-central1)...
i functions: updating Node.js 10 function groupEvents(us-central1)...
i functions: updating Node.js 10 function addUserToGroup(us-central1)...
i functions: updating Node.js 10 function unsubscribeUserFromTopicOnDelete(us-central1)...
i functions: updating Node.js 10 function participantRemoved(us-central1)...
i functions: updating Node.js 10 function groupAdminChanged(us-central1)...
i functions: updating Node.js 10 function groupInfoChanged(us-central1)...
i functions: updating Node.js 10 function deleteMessageForGroup(us-central1)...
i functions: updating Node.js 10 function deleteMessageForBroadcast(us-central1)...
i functions: updating Node.js 10 function deleteMessage(us-central1)...
i functions: updating Node.js 10 function saveUidOnLogin(us-central1)...
i functions: updating Node.js 10 function getTime(us-central1)...
i functions: updating Node.js 10 function setStatusCount(us-central1)...
i functions: updating Node.js 10 function subscribeToBroadcast(us-central1)...
i functions: updating Node.js 10 function resubscribeUserToBroadcasts(us-central1)...
i functions: updating Node.js 10 function sendMessageToBroadcast(us-central1)...
i functions: updating Node.js 10 function unsubscribeUserFromBroadcast(us-central1)...
i functions: updating Node.js 10 function sendUnDeliveredNotifications(us-central1)...
i functions: updating Node.js 10 function sendNewCallNotification(us-central1)...
i functions: updating Node.js 10 function indexNewCall(us-central1)...
i functions: updating Node.js 10 function indexNewGroupCall(us-central1)...
i functions: updating Node.js 10 function indexPKToken(us-central1)...
i functions: updating Node.js 10 function deviceIdChanged(us-central1)...
i functions: updating Node.js 10 function getVirgilJwt(us-central1)...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/subscribeToBroadcast. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/setStatusCount. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/groupInfoChanged. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/participantAdded. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/indexNewGroupCall. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/getVirgilJwt. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/deleteMessageForGroup. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/unsubscribeUserFromBroadcast. Waiting to retry...
i functions: updating Node.js 10 function subscribeToBroadcast(us-central1)...
i functions: updating Node.js 10 function setStatusCount(us-central1)...
i functions: updating Node.js 10 function groupInfoChanged(us-central1)...
i functions: updating Node.js 10 function participantAdded(us-central1)...
i functions: updating Node.js 10 function indexNewGroupCall(us-central1)...
i functions: updating Node.js 10 function getVirgilJwt(us-central1)...
i functions: updating Node.js 10 function deleteMessageForGroup(us-central1)...
i functions: updating Node.js 10 function unsubscribeUserFromBroadcast(us-central1)...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/groupInfoChanged. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/participantAdded. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/setStatusCount. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/getVirgilJwt. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/subscribeToBroadcast. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/deleteMessageForGroup. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/indexNewGroupCall. Waiting to retry...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/unsubscribeUserFromBroadcast. Waiting to retry...
i functions: updating Node.js 10 function groupInfoChanged(us-central1)...
i functions: updating Node.js 10 function participantAdded(us-central1)...
i functions: updating Node.js 10 function setStatusCount(us-central1)...
i functions: updating Node.js 10 function getVirgilJwt(us-central1)...
i functions: updating Node.js 10 function subscribeToBroadcast(us-central1)...
i functions: updating Node.js 10 function deleteMessageForGroup(us-central1)...
i functions: updating Node.js 10 function indexNewGroupCall(us-central1)...
i functions: updating Node.js 10 function unsubscribeUserFromBroadcast(us-central1)...
⚠ functions: got "Quota Exceeded" error while trying to update projects/ichat-daac0/locations/us-central1/functions/indexNewGroupCall. Waiting to retry...
i functions: updating Node.js 10 function indexNewGroupCall(us-central1)...
Functions deploy had errors with the following functions:
sendMessageNotification(us-central1)
saveUidOnLogin(us-central1)
groupEvents(us-central1)
sendNewCallNotification(us-central1)
indexNewCall(us-central1)
addUserToGroup(us-central1)
resubscribeUserToBroadcasts(us-central1)
deviceIdChanged(us-central1)
deleteMessage(us-central1)
sendUnDeliveredNotifications(us-central1)
sendMessagesForGroups(us-central1)
deleteMessageForBroadcast(us-central1)
unsubscribeUserFromTopicOnDelete(us-central1)
indexPKToken(us-central1)
sendMessageToBroadcast(us-central1)
groupAdminChanged(us-central1)
getTime(us-central1)
participantRemoved(us-central1)
participantAdded(us-central1)
subscribeToBroadcast(us-central1)
groupInfoChanged(us-central1)
getVirgilJwt(us-central1)
unsubscribeUserFromBroadcast(us-central1)
deleteMessageForGroup(us-central1)
setStatusCount(us-central1)
indexNewGroupCall(us-central1)
To try redeploying those functions, run:
firebase deploy --only "functions:sendMessageNotification,functions:saveUidOnLogin,functions:groupEvents,functions:sendNewCallNotification,functions:indexNewCall,functions:addUserToGroup,functions:resubscribeUserToBroadcasts,functions:deviceIdChanged,functions:deleteMessage,functions:sendUnDeliveredNotifications,functions:sendMessagesForGroups,functions:deleteMessageForBroadcast,functions:unsubscribeUserFromTopicOnDelete,functions:indexPKToken,functions:sendMessageToBroadcast,functions:groupAdminChanged,functions:getTime,functions:participantRemoved,functions:participantAdded,functions:subscribeToBroadcast,functions:groupInfoChanged,functions:getVirgilJwt,functions:unsubscribeUserFromBroadcast,functions:deleteMessageForGroup,functions:setStatusCount,functions:indexNewGroupCall"
To continue deploying other features (such as database), run:
firebase deploy --except functions
Error: Functions did not deploy properly.
mac@bogon functions %
This is also happening for us pretty consistently, 3-5 will fail with quota exceeded when deploying 60 functions. We're using firebase-tools 9.23.0.
HOW do you solve this issue? Tried to deploy one by one and still getting errors.
To try redeploying those functions, run: firebase deploy --only "functions:sendMessageNotification,functions:groupAdminChanged,functions:deleteMessageForGroup,functions:setStatusCount,functions:groupInfoChanged,functions:deleteMessageForBroadcast,functions:unsubscribeUserFromBroadcast,functions:unsubscribeUserFromTopicOnDelete,functions:saveUidOnLogin,functions:indexNewGroupCall,functions:sendMessagesForGroups,functions:addUserToGroup,functions:participantRemoved,functions:participantAdded,functions:groupEvents,functions:deleteMessage,functions:getVirgilJwt,functions:sendMessageToBroadcast,functions:indexPKToken,functions:sendNewCallNotification,functions:subscribeToBroadcast,functions:indexNewCall,functions:sendUnDeliveredNotifications,functions:deviceIdChanged,functions:resubscribeUserToBroadcasts,functions:getTime"
To continue deploying other features (such as database), run: firebase deploy --except functions
Error: Functions did not deploy properly. mac@bogon functions % firebase deploy --only "functions:getTime"
=== Deploying to 'ichat-daac0'...
i deploying functions i functions: ensuring required API cloudfunctions.googleapis.com is enabled... i functions: ensuring required API cloudbuild.googleapis.com is enabled... ✔ functions: required API cloudbuild.googleapis.com is enabled ✔ functions: required API cloudfunctions.googleapis.com is enabled i functions: preparing functions directory for uploading... i functions: packaged functions (73.93 KB) for uploading ✔ functions: functions folder uploaded successfully i functions: updating Node.js 10 function getTime(us-central1)...
Functions deploy had errors with the following functions: getTime(us-central1)
To try redeploying those functions, run: firebase deploy --only "functions:getTime"
To continue deploying other features (such as database), run: firebase deploy --except functions
Error: Functions did not deploy properly. mac@bogon functions % functions % firebase deploy --only "functions:resubscribeUserToBroadcasts" mac@bogon functions % firebase deploy --only "functions:resubscribeUserToBroadcasts"
=== Deploying to 'ichat-daac0'...
i deploying functions i functions: ensuring required API cloudfunctions.googleapis.com is enabled... i functions: ensuring required API cloudbuild.googleapis.com is enabled... ✔ functions: required API cloudfunctions.googleapis.com is enabled ✔ functions: required API cloudbuild.googleapis.com is enabled i functions: preparing functions directory for uploading... i functions: packaged functions (73.93 KB) for uploading ✔ functions: functions folder uploaded successfully i functions: updating Node.js 10 function resubscribeUserToBroadcasts(us-central1)...
Functions deploy had errors with the following functions: resubscribeUserToBroadcasts(us-central1)
To try redeploying those functions, run: firebase deploy --only "functions:resubscribeUserToBroadcasts"
To continue deploying other features (such as database), run: firebase deploy --except functions
Error: Functions did not deploy properly.
Having trouble? Try firebase [command] --help
Hey all - looks like there are a few different issues here, so I'll take them one at a time:
- To the original issue of deployments of a large number of functions hitting quota errors and failing: As @abeisgoat mentioned, we are aware of the problems encountered when deploying lots of functions at once. We're currently working on some improvements to the throttler logic that ought to help - however, we strongly recommend making small groups of functions and deploying them in smaller groups. https://firebase.google.com/docs/functions/organize-functions#group_functions has a good guide on how to do this.
- To the
i functions: cleaning up build files... Cleaning up project directory and file based variables 00:01 ERROR: Job failed: exit code 1error message: This message doesn't affect the success or failure of your function deployments - after a deployment is finished, the CLI tries to clean up some build artifacts from Google Container Registry to save some storage space. - @yendiscart It looks like you are experiencing a different issue than the one described here. Could you retry your deployment with the --debug flag (
firebase deploy --only "functions:resubscribeUserToBroadcasts" --debugand open a new issue including the output of that command?
@joehan The original problem that relates with the large number of functions is the same as I'm facing? Because I have a similar issue that outputs a quota error although I'm trying to deploy around 20 functions only. I'm facing:
{"@type":"type.googleapis.com/google.cloud.audit.AuditLog","status":{"code":8,"message":"Build failed: Quota exceeded for quota metric 'Build Create requests' and limit 'Build Create requests per minute' of service 'cloudbuild.googleapis.com' for consumer 'project_number:<NUMBER>'."},"authenticationInfo":{"principalEmail":"<EMAIL>"},"serviceName":"cloudfunctions.googleapis.com","methodName":"google.cloud.functions.v1.CloudFunctionsService.UpdateFunction","resourceName":"projects/ticker-dev-4d59e/locations/southamerica-east1/functions/<FUNCTION_NAME>"}
For clarification, are these failures longstanding issues or regressions as of a recent version of firebase-tools? We did some major refactoring recently and it's possible we have a regression for large codebases
My error was solved by upgrading my billing or plan to Blaze before I could deploy functions.
HOW do you solve this issue? Tried to deploy one by one and still getting errors.
To try redeploying those functions, run: firebase deploy --only "functions:sendMessageNotification,functions:groupAdminChanged,functions:deleteMessageForGroup,functions:setStatusCount,functions:groupInfoChanged,functions:deleteMessageForBroadcast,functions:unsubscribeUserFromBroadcast,functions:unsubscribeUserFromTopicOnDelete,functions:saveUidOnLogin,functions:indexNewGroupCall,functions:sendMessagesForGroups,functions:addUserToGroup,functions:participantRemoved,functions:participantAdded,functions:groupEvents,functions:deleteMessage,functions:getVirgilJwt,functions:sendMessageToBroadcast,functions:indexPKToken,functions:sendNewCallNotification,functions:subscribeToBroadcast,functions:indexNewCall,functions:sendUnDeliveredNotifications,functions:deviceIdChanged,functions:resubscribeUserToBroadcasts,functions:getTime"
To continue deploying other features (such as database), run: firebase deploy --except functions
Error: Functions did not deploy properly. mac@bogon functions % firebase deploy --only "functions:getTime"
=== Deploying to 'ichat-daac0'...
i deploying functions i functions: ensuring required API cloudfunctions.googleapis.com is enabled... i functions: ensuring required API cloudbuild.googleapis.com is enabled... ✔ functions: required API cloudbuild.googleapis.com is enabled ✔ functions: required API cloudfunctions.googleapis.com is enabled i functions: preparing functions directory for uploading... i functions: packaged functions (73.93 KB) for uploading ✔ functions: functions folder uploaded successfully i functions: updating Node.js 10 function getTime(us-central1)...
Functions deploy had errors with the following functions: getTime(us-central1)
To try redeploying those functions, run: firebase deploy --only "functions:getTime"
To continue deploying other features (such as database), run: firebase deploy --except functions
Error: Functions did not deploy properly. mac@bogon functions % functions % firebase deploy --only "functions:resubscribeUserToBroadcasts" mac@bogon functions % firebase deploy --only "functions:resubscribeUserToBroadcasts"
=== Deploying to 'ichat-daac0'...
i deploying functions i functions: ensuring required API cloudfunctions.googleapis.com is enabled... i functions: ensuring required API cloudbuild.googleapis.com is enabled... ✔ functions: required API cloudfunctions.googleapis.com is enabled ✔ functions: required API cloudbuild.googleapis.com is enabled i functions: preparing functions directory for uploading... i functions: packaged functions (73.93 KB) for uploading ✔ functions: functions folder uploaded successfully i functions: updating Node.js 10 function resubscribeUserToBroadcasts(us-central1)...
Functions deploy had errors with the following functions: resubscribeUserToBroadcasts(us-central1)
To try redeploying those functions, run: firebase deploy --only "functions:resubscribeUserToBroadcasts"
To continue deploying other features (such as database), run: firebase deploy --except functions
Error: Functions did not deploy properly.
Having trouble? Try firebase [command] --help
Unfortunately I am also experiencing these issue often (if not each time) trying to deploy all my GCF at once
⚠ functions: got "Quota Exceeded" error while trying to update *********** Waiting to retry...
I know it's best to deploy only the group of functions that have been updated and leave the other ones untouched, but sometimes when updating environment configuration it is required to deploy all cloud function to allow the new configuration to propagate properly.
Please advise on this issue, it would be great to find a better workaround to allow better integration with CI too.
Thanks!
Experiencing same issue while deploying 62 functions.
⚠ functions: got "Quota Exceeded" error while trying to update projects/<redacted>/locations/us-central1/functions/<redacted>. Waiting to retry...
2 functions fails to deploy, thus I assume there should be somewhere 60 function deployment limit?
As @42ae mentioned previously, sometimes bulk deployment is needed in order for all functions to be updated. Looking forward hearing for solutions to this problem.
We are also experiencing something similar with about 80 functions where 5-6 regularly fail. We are currently on version 10.1.4.
functions: got "Quota Exceeded" error while trying to update projects/***/locations/northamerica-northeast1/functions/<function>. Waiting to retry...
I am frequently experiencing this issue as well while deploying over 100 functions. We deploy in groups and have created some parsing logic for our CI system, however it would be very convenient to have these auto-retry if we pass the deployment quota, or at least spit out a command after failure to deploy the rest of the functions (currently it just lists them).
The current text says "waiting to retry" while we exceed the quota, so that was the behavior I was expecting, however it is not what ended up happening.
I can consistently get this error even when deploying a group of only 5 functions. The error message literally shows nothing useful and it has been a real pain. I can deploy each one individually which results in deployments for a full stack to take hours. Any ideas from folks on how to overcome this would be really appreciated.
$ firebase --version
10.5.0
$ gcloud --version
Google Cloud SDK 369.0.0
alpha 2022.01.14
beta 2022.01.14
bq 2.0.72
core 2022.01.14
gsutil 5.6
Deployment failure:
Build failed: {"metrics":{},"error":{"buildpackId":"","buildpackVersion":"","errorType":"OK","canonicalCode":"OK","errorId":"","errorMessage":""},"stats":[{"buildpackId":"google.utils.archive-source","buildpackVersion":"0.0.1","totalDurationMs":199,"userDurationMs":198},{"buildpackId":"google.nodejs.npm","buildpackVersion":"0.9.0","totalDurationMs":19905,"userDurationMs":19841},{"buildpackId":"google.nodejs.functions-framework","buildpackVersion":"0.9.4","totalDurationMs":3764,"userDurationMs":3744},{"buildpackId":"google.utils.label","buildpackVersion":"0.0.2","totalDurationMs":0,"userDurationMs":0}],"warnings":null,"customImage":false}
EDIT: 4/7/2022
This issue seemed to only be happening in Github Actions CI on a cloud runner. It seemed perhaps the runner was running out of memory. I modified the runner to a self-hosted runner and the issues seemed to go away. Hopefully this helps someone else.
We will prioritize looking into this soon. There will be a three pronged strategy:
- Fixing the "invalid source token" problem, which occurs when function deploys take a very long time
- Looking at tuning the rate limiter to avoid quota errors
- Improving deploy code so that it can recognize and skip functions that don't need redeployment
Hopefully this will help you all when we can get these improvements ready.
We've currently launched (but not well documented or added wizzard support for) codebases. With codebases you can have multiple directories of Cloud Functions. Simply create a firebase.json fragment that looks a bit like:
"functions" : [
{
"source": "functions",
"codebase": "main"
}, {
"source": "otherfunctions",
"codebase": "other"
}
]
With this, you can now call firebase deploy functions:main or firebase deploy functions:other. Next, we're working on recognizing that a codebase hasn't been modified since last deployment and we'll skip those functions altogether.
@inlined is it possible there is a regression and https://github.com/firebase/firebase-tools/issues/2606 / https://github.com/firebase/firebase-tools/pull/3246 no longer works? Is it possible to restore 'retry on quota' functionality?
We've currently launched (but not well documented or added wizzard support for) codebases. With codebases you can have multiple directories of Cloud Functions. Simply create a firebase.json fragment that looks a bit like:
"functions" : [ { "source": "functions", "codebase": "main" }, { "source": "otherfunctions", "codebase": "other" } ]With this, you can now call
firebase deploy functions:mainorfirebase deploy functions:other. Next, we're working on recognizing that a codebase hasn't been modified since last deployment and we'll skip those functions altogether.
@inlined Sorry, but I don't see how introducing codebases aims to solve this issue (what if a single codebase has more functions than the quota limit allows?). While it might become useful in some cases it certainly won't be helpful for me.
Wouldn't it be better if there was an option to deploy only modified functions? (--only-changed?) Usually a single push only affects a few functions so deploying only those would help.
I'm getting "Invaild source token" every time for the same 2 functions when deploying 24 Node.js v16 functions at once. If I deploy then individually, there's no error. I'm using [email protected].
I've sent a bug report through https://firebase.google.com/support/troubleshooter/report/bugs and they answered pointing to this part of the documentation:
When deploying large numbers of functions, you may exceed the standard quota and receive HTTP 429 or 500 error messages. To solve this, deploy functions in groups of 10 or fewer. Link: https://firebase.google.com/docs/functions/manage-functions?authuser=0#deploy_functions
But @taeold said in another issue that:
The Firebase CLI is tuned to deploy in batches of 40 functions at any given time, so I suspect issuing a single firebase deploy command to deploy all your 80+ functions should be much better than your expectation. Link: https://github.com/firebase/firebase-tools/issues/4266#issuecomment-1067381741
They seem to contradict each other. What am I missing?
So, I was trying to deploy 24 functions at once and 2 failed. I've experimented more and realized that if I deploy exactly those 2 problematic functions, deployment happens just fine. If I deploy the 2 problematic functions + another function, the 2 problematic functions fail again while the other function is successfully deployed.
EDIT: Just upgraded to [email protected] and deploying the 24 functions at once works again!
Scanning through some bugs and came across this one. I wanted to give a few updates:
- Regression/Fix. I'm glad that the regression seems fixed already and sorry that we had a regression.
- Invalid source token. We have not started a fix here. If someone wants to try fixing this, it requires some cleverness but not knowledge of the codebase
- Quotas You may find some (seemingly) contradictory advice about running w/ --only and groups of 10 functions or that we may run your functions in batches. The Firebase CLI already batches your functions for deploy thanks to the ThrottlerQueue class; this is basically a JS equivalent of a threadpool and makes sure we only have N functions deploying concurrently. Then with backoffs we approximate the most calls you can make in a project within quota limits. It's a pipe dream to have an improved request queue that is quota aware so it can do smarter backoffs, but we may not get there for a long time. Manually deploying only a few functions at a time is even more conservative than normal. You are not supposed to run that command several times in parallel, since that removes all benefits of fewer deploys. In the future, Codebases will also help you limit the number of functions that are deployed.
- Codebases feature
Wouldn't it be better if there was an option to deploy only modified functions? (--only-changed?) Usually a single push only affects a few functions so deploying only those would help.
@Kopunk Unfortunately, a more accurate implementation of --only-changed that isn't the codebases feature would be about as complicated as writing a JavaScript interpreter (which means that a true implementation of --only-changed is probably also impossible due to the Halting Problem). Any non-trivial cloud function leans on utility methods. Any change to any one of these utility methods will change some amount of code that can only be determined by running through a program.
Codebases is a compromise. If you agree to put firewalls between parts of your code (e.g. directories) we can instead do a file diff and notice nothing changes. Sometimes you may have shared code and it truly isn't possible to separate your code cleanly (e.g. there are shared interfaces). For the first several years of its existence, Firebase solved this problem using an internal-API node module. I've heard that Yarn workspaces can solve this quite eloquently as well. Many teams at Google use them.
We've got a large number of functions (~150) we deploy regularly. We get maybe 1 - 2 failed function deploys per run due to the "Quota Exceeded" error.
I thought firebase-tools was meant to manage this & retry, but we're not seeing this logic succeed.
Is there any way to work around this without having to rearchitect our entire production application? I understand the codebases pattern, but this won't work for our use case given all the functions are camel-case named.
@willhackett I'd love to understand what you mean by camlcased name functions better - can you elaborate why codebases doesn't fit your application?
I'm planning to review https://github.com/firebase/firebase-tools/pull/4967 soon, but my feeling is that fighting the GCF quota limit isn't fool-proof way, especially since quota is different depending on your project's state, location, history, etc.
Our thesis is that codebase with feature to automcatically skip deployment of function that doesn't require a re-deploy is more promising and I'd love to understand in what ways we could improve codebase support to work more developers' setups.
@taeold Suppose I want to have very granular updates. If a single function changed, I want to redeploy just that one function and no other. Would it make sense to have 1 function per codebase? Any possible issues? Or do you think that would be over-engineering in the first place?
@gustavopch I think somewhere between the 2 extremes (1 codebase 100 functions vs 100 codebase 1 function) would help. For example, you might group function in a codebase by trigger type. Or by their business domain. In the ideal case, when you are apply a bug fix or a new feature, it would affect a single codebase. But I suspect that our reality is a bit messier than that.
You can give "skip deploying a codebase that's doesn't require one" right now by using the latest Firebase CLI and enabling the priview:
firebase --open-sesame skipdeployingnoopfunctions
We are making some last pushes to get this feature cleaned up, an when its ready you see it be mentioned in our documentations. Would love to hear from advanced users like yourselves what you think.