firebase-tools

firebase-tools copied to clipboard

[REQUIRED] Environment info

firebase-tools: 11.8.0 (latest release)

Platform: : macOS

[REQUIRED] Test case

If I make a function on the index.js file, and then I try to deploy using: firebase deploy --only functions:nameOfYourFunction I get an Unexpected Error at the end.

[REQUIRED] Steps to reproduce

Using npm or whatever package manager you prefer, run these on your terminal,

npm install -g firebase-tools firebase login firebase init functions

Now under the functions folder, make sure you have installed node modules using npm install, and finally you create a basic function you like on index.js after you are done importing stuffs, for example:

const functions = require('firebase-functions'); const admin = require('firebase-admin'); admin.initializeApp();

exports.yourFunctionName = functions.https.onRequest(async(req,res)=>{ functions.logger.log(res.body); //just for an example. })

Now when you try to do:

firebase deploy --only functions:yourFunctionName

You'll see this error message in the end:

[debug] [2022-09-05T11:21:08.962Z] Building nodejs source [debug] [2022-09-05T11:21:08.963Z] Could not find functions.yaml. Must use http discovery [debug] [2022-09-05T11:21:08.972Z] Error: spawn ./node_modules/.bin/firebase-functions ENOENT at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19) at onErrorNT (node:internal/child_process:476:16) at processTicksAndRejections (node:internal/process/task_queues:83:21) [error] [error] Error: An unexpected error has occurred. [debug] [2022-09-05T11:21:09.063Z] Serving at port 9005

[REQUIRED] Expected behavior

Expected behaviour is that the function gets deployed successfully without any error.

[REQUIRED] Actual behavior

Actual behaviour is that function normally get's deployed successfully and just after this update, this issue came into play.

I could fix this issue just by downgrading the version of firebase tools to v10.0.0 using: npm install -g [email protected]

Since there might be something wrong with the new v11.8.0, so It would be really awesome if the engineers took a look at what's up with the latest release...so I am keeping this issue open.

Thanks for filing the issue @rahuld600!

QQ1 - what tooling are you using for package management (yarn? npm)? Are you usign yarn/npm workspaces? QQ2 - have you run npm i before deploying your function?

@taeold Hello sir, First of all, thank you for checking out! QQ1 - I am using npm for package management. QQ2 - Yes sir! I ran npm install before deploying.

Thanks for filing the issue @[removed to avoid tagging]!

QQ1 - what tooling are you using for package management (yarn? npm)? Are you usign yarn/npm workspaces? QQ2 - have you run npm i before deploying your function?

Hey. If I may, what if they weren't using npm but yarn and yarn workspaces? We seem to be having the same issue, only with yarn workspaces and for firebase-tools v11.15.0.

Would it be better if I created a new issue for this for full details, or is this fine? Here are some of the other relevant(?) packages and versions: "firebase": "9.12.1" "firebase-admin": "^11.2.0" "firebase-functions": "4.0.1"

Hey :wave:

I'm seeing a similarish issue with firebase-tools v11.15.0 when running firebase deploy --only functions --debug

debug output:

[2022-10-29T06:03:49.031Z] Building nodejs source
[2022-10-29T06:03:49.032Z] Could not find functions.yaml. Must use http discovery
[2022-10-29T06:03:49.195Z] Serving at port 9005

[2022-10-29T06:03:49.372Z] Got response from /__/functions.yaml Failed to generate manifest from function source: Error
[2022-10-29T06:03:49.381Z] shutdown requested via /__/quitquitquit

Error: Failed to parse build specification:
- FirebaseError Expect manifest yaml to specify a version number

I deleted the existing functions as I thought it would have something do with the versioning but no dice.

package.json contents:

{
  "name": "functions",
  "scripts": {},
  "engines": {
    "node": "16"
  },
  "main": "lib/index.js",
  "dependencies": {
    "@google-cloud/firestore": "6.4.1",
    "axios": "1.1.3",
    "firebase-admin": "11.2.0",
    "firebase-functions": "4.0.1",
    "ksuid": "3.0.0",
    "purify-ts": "1.3.0",
    "stripe": "10.15.0",
    "uuid": "9.0.0"
  },
  "devDependencies": {
    "firebase-functions-test": "3.0.0",
    "typescript": "4.8.4"
  },
  "private": true
}

I have this bug on 11.16.0, the error I see (Node.js version v14.21.1):

[debug] [2022-11-10T05:02:40.798Z] Building nodejs source
[info] i  functions: Loaded environment variables from .env. 
[debug] [2022-11-10T05:02:40.800Z] Could not find functions.yaml. Must use http discovery
[debug] [2022-11-10T05:02:40.807Z] Error: spawn ./node_modules/.bin/firebase-functions ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:277:19)
    at onErrorNT (internal/child_process.js:472:16)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)
[error] 
[error] Error: An unexpected error has occurred.

@vevepeter here's a feature request issue for mono-repos - #653

@joonne that's a different error than what the other folks are experiencing, do you mind opening another issue and use the template? thanks

@kossnocorp are you also using vanilla npm or another package manager?

@colerogers I use npm. I found a solution - upgrading from Node v14 to v16 helped to solve the problem.

I could fix this issue just by downgrading the version of firebase tools to v10.0.0 using: npm install -g [email protected]

This version npm install -g [email protected] also works but higher than this version does not work atleast for me

I have the same problem with my Nuxt app. To avoid it, I delete the node_models folder in the output directory and reinstall it.

$ NITRO_PRESET=firebase npm run build && cd .output/server && rm -rf node_modules && npm i && cd ../../ && firebase login && firebase deploy

After that everything works normally.

Node v16.18.1 Firebase v11.16.1 Ubuntu 22.04

Just rolling back to 10.8.0 has solved it for me, lost 2 days of debugging a pipeline haha

Updating firebase tool from 11.2.2 → 11.17.0 worked for me.. Spend 3 days on this. Randomly break my project due to this

Hi everyone sorry for the delay, can anyone confirm they are still seeing this exact issue in the latest release (11.18.0)? I'm not able to reproduce this on my end. Thanks

Definitely got the issue using the latest 11.19.0

Downgrading to 11.17.0 has fixed the issue.

I'm also seeing the same on 11.19.0 but was working fine on 11.18.0. Also happens when running emulators:start and not just deploy. I've found the code which it differentiates... running firebase emulators:start --only functions --debug on 11.18.0 outputs:

[2022-12-21T03:49:26.597Z] Validating nodejs source
[2022-12-21T03:49:26.923Z] checkFunctionsSDKVersion encountered error:Invalid Version:  {}
[2022-12-21T03:49:26.924Z] > [functions] package.json contents: {
  "name": "functions",
  "engines": {
    "node": "16"
  },
  "private": true,
  "type": "module",
  "version": "5.2.0",
  "dependencies": {
    "@google-cloud/functions-framework": "3.1.2",
    "array-flatten": "3.0.0",
    "firebase": "9.15.0",
    "firebase-admin": "11.4.0",
    "firebase-functions": "4.1.1",
    "lodash.merge": "4.6.2",
    "node-fetch": "3.3.0",
    "nodemailer": "6.8.0"
  }
}
[2022-12-21T03:49:26.924Z] Building nodejs source
[2022-12-21T03:49:26.925Z] Analyzing nodejs backend spec
[2022-12-21T03:49:27.523Z] Could not parse firebase-functions version '' into semver. Falling back to parseTriggers.
✔  functions: Loaded functions definitions from source: ...

Running on v11.19.0 outputs:

[2022-12-21T04:03:24.281Z] Validating nodejs source
[2022-12-21T04:03:24.585Z] > [functions] package.json contents: {
  "name": "functions",
  "engines": {
    "node": "16"
  },
  "private": true,
  "type": "module",
  "version": "5.2.0",
  "dependencies": {
    "@google-cloud/functions-framework": "3.1.2",
    "array-flatten": "3.0.0",
    "firebase": "9.15.0",
    "firebase-admin": "11.4.0",
    "firebase-functions": "4.1.1",
    "lodash.merge": "4.6.2",
    "node-fetch": "3.3.0",
    "nodemailer": "6.8.0"
  }
}
[2022-12-21T04:03:24.586Z] Building nodejs source
[2022-12-21T04:03:24.591Z] Could not find functions.yaml. Must use http discovery
[2022-12-21T04:03:24.600Z] Error: spawn ./node_modules/.bin/firebase-functions ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:285:19)
    at onErrorNT (node:internal/child_process:485:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

Error: An unexpected error has occurred

So you can see the working version says "Could not parse firebase-functions version '' into semver. Falling back to parseTriggers" which is logged from here

Which is only happening when this.sdkVersion is not valid... v11.18.0: this.sdkVersion === "" v11.19.0: this.sdkVersion === 4.1.1 Change happened here - https://github.com/firebase/firebase-tools/commit/355e9b986ed4c947757666c08b128cf22e5cb092#diff-88e84b107a8f48a6a783275f2cef81ea7f269efbc4d2c22906217d0be811d973

This is the cause of then hitting let discovered = await discovery.detectFromYaml(this.sourceDir, this.projectId, this.runtime); and that function tries to rad the functions.yaml file which is not found causing that message:

// https://github.com/firebase/firebase-tools/blob/355e9b986ed4c947757666c08b128cf22e5cb092/src/deploy/functions/runtimes/discovery/index.ts#L43
try {
        text = await exports.readFileAsync(path.join(directory, "functions.yaml"), "utf8");
    }
    catch (err) {
        if (err.code === "ENOENT") {
            logger_1.logger.debug("Could not find functions.yaml. Must use http discovery");
        }
        else {
            logger_1.logger.debug("Unexpected error looking for functions.yaml file:", err);
        }
        return;
    }

Ahh, digging deeper where the sdkVersion is set in getFunctionsSDKVersion I discovered I do not install node_modules directly inside my functions dir (they are duplicated in the parent dir and node finds them there).

Once I installed with npm i from within the functions dir the output was more similar to v11.19.0 complaining about not finding functions.yaml.

[2022-12-21T04:26:40.814Z] Building nodejs source
[2022-12-21T04:26:40.814Z] Analyzing nodejs backend spec
SDK VERSION 4.1.1
[2022-12-21T04:26:40.815Z] Could not find functions.yaml. Must use http discovery
[2022-12-21T04:26:40.950Z] Serving at port 9005

But the difference here is this does manage to serve successfully but v11.19.0 errors out. Back to the 🐞 hunt...

I can confirm that running npm install from within the functions dir did allow me to run firebase emulators:start --only functions 🎉

I'm not sure about others here but my project is written in TS and a build step outputs JS to functions dir, but does not copy over or install deps from that package.json. Quick solution for now is to add another task to my build or test scripts to ensure npm install happens inside that dir.

Seems as though the way firebase-tools determines the sdkVersion changed a lot to work with yarn workspaces https://github.com/firebase/firebase-tools/commit/355e9b986ed4c947757666c08b128cf22e5cb092#diff-0342df3e11dd503fdca5b2e49395567874f9a9438f96eb24431773d8465db01f

The previous way (using spawn.sync("npm", ["list", "firebase-functions", "--json=true"]) relied on npm's resolver to keep looking upward and in my case always found the dep installed in the parent dir.

Tagging @taeold as the dev who committed that change - hopefully this thread will help others and we can get a more robust solution 🙏

I just upgraded from 11.14.3 to 11.19.0 and am getting the error when running in an emulator in a Docker container. My project is also written in TS.

I rolled back to 11.17.0 and it works as a workaround for now.

firebase-debug.log

[debug] [2022-12-21T06:37:26.858Z] Validating nodejs source
[debug] [2022-12-21T06:37:27.225Z] > [functions] package.json contents: {
  "dependencies": {
    "@angular/animations": "14.2.12",
    "@angular/common": "14.2.12",
    "@angular/core": "14.2.12",
    "@angular/platform-browser": "14.2.12",
    "@angular/router": "14.2.12",
    "@dereekb/date": "9.20.2",
    "@dereekb/dbx-core": "9.20.2",
    "@dereekb/firebase": "9.20.2",
    "@dereekb/firebase-server": "9.20.2",
    "@dereekb/model": "9.20.2",
    "@dereekb/nestjs": "9.19.2",
    "@dereekb/rxjs": "9.20.2",
    "@dereekb/util": "9.20.2",
    "@nestjs/common": "9.0.9",
    "@nestjs/config": "^2.0.1",
    "class-validator": "^0.13.2",
    "extra-set": "^2.2.11",
    "firebase": "9.15.0",
    "firebase-admin": "11.4.0",
    "firebase-functions": "4.1.1",
    "form-data": "4.0.0",
    "lodash.isequal": "^4.5.0",
    "mailgun.js": "^8.0.0",
    "make-error": "^1.3.0",
    "reflect-metadata": "^0.1.13",
    "rxjs": "7.5.6",
    "ts-essentials": "^9.1.2",
    "tslib": "^2.0.0",
    "uuid": "^9.0.0",
    "zone.js": "~0.11.8"
  },
  "main": "main.js"
}
[debug] [2022-12-21T06:37:27.225Z] Building nodejs source
[debug] [2022-12-21T06:37:27.232Z] Could not find functions.yaml. Must use http discovery
[debug] [2022-12-21T06:37:27.244Z] Error: spawn ./node_modules/.bin/firebase-functions ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:285:19)
    at onErrorNT (node:internal/child_process:485:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
[error] 
[error] Error: An unexpected error has occurred.

@dereekb did you ensure your node modules are installed in your functions dir? That's what did it for me

@ChromeQ Don't have time to dig into the issue right now, but I will try that when I get a chance! I'll have to think about the setup a bit more since right now I'm using Nx and Docker and running an additional npm install into the output functions directory seems like something I'd rather not add as another step when running locally for development. although for deploying to Firebase it isn't a big deal to add that step to the CI.

11.19.0 isn't a critical version for me right now so I'll wait and see what happens and cross that road when I have to.

That's fair. But if you already have an npm cache populated from another install then it won't take long at all, or rsync or cp the whole node modules or even symlink might work. At least for me as all I needed it for was the tests cos firebase will run the install anyway when you deploy

Just confirming that my project is set up as a monorepo with yarn workspaces, without the nmHoistingLimits: workspace setting in my .yarnrc.yml configuration (for better or worse), so there is no node_modules folder in my function source. Functions written in TS and have a predeploy script in firebase.json to build to JS. I get the same errors mentioned in this thread with v11.19.0. v11.18.0 is fine.

@myktra if you cd into your functions dir (which has no node_modules) and run npm I then did the error go away?

OK, that seemed to work.

By making a change to my predeploy script in firebase.json – adding npm --prefix \"functions/up\" i – I managed to get v11.19.0 working:

"functions": {
  "predeploy": [
    "npm --prefix \"functions/up\" i && npm --prefix \"functions\" run build"
  ],
  "source": "functions/up"
}

The firebase part of my monorepo sorta looks like this:

/services/firebase (firebase.json here)
/services/firebase/functions, /services/firebase/functions/src (TS function source here)
/services/firebase/functions/up (tsc/webpack-compiled JS here)

I'd prefer not to have to do this - would seem to require additional storage and take more time to execute since I'm using yarn as my package manager which has already downloaded and cached the necessary packages. But indeed this workaround would get things working with v11.19.0. I'd prob revert to v11.18.0 until the previous behavior was restored in v11.19+, assuming that is generally more desirable.

Hey @rahuld600. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

I can confirm this (11.19.0) also broke my setup too.

I can confirm that running npm install from within the functions dir did allow me to run firebase emulators:start --only functions 🎉

I'm not sure about others here but my project is written in TS and a build step outputs JS to functions dir, but does not copy over or install deps from that package.json. Quick solution for now is to add another task to my build or test scripts to ensure npm install happens inside that dir.

Seems as though the way firebase-tools determines the sdkVersion changed a lot to work with yarn workspaces 355e9b9#diff-0342df3e11dd503fdca5b2e49395567874f9a9438f96eb24431773d8465db01f

The previous way (using spawn.sync("npm", ["list", "firebase-functions", "--json=true"]) relied on npm's resolver to keep looking upward and in my case always found the dep installed in the parent dir.

Tagging @taeold as the dev who committed that change - hopefully this thread will help others and we can get a more robust solution 🙏

Yup, this is likely the issue. @taeold can you take a look at this? I'm using Nx workspace, in which I'm targeting my functions source directory to the compilation output of Nx. For example:

    {
      "source": "apps/server/dist",
      "predeploy": [
      ],
      "runtime": "nodejs16",
      "codebase": "server"
    },

I believe the commit @ChromeQ mentioned is why this no longer works. I don't think installing dependencies in Nx projects/apps is encouraged behavior, so hopefully, there are ways we can support both the intended fix of that commit and cross-compatibility with other systems (like Nx).

Hi folks - thanks for escalating this issue. Will make this my priority today.

Just finished reading through the issue - thanks @rahuld600 @ChromeQ @dereekb @michaelangrivera and everyone else for sharing your debugging notes.

For context here's quick summary of how function deployment works:

Firebase CLI needs to know the functions are defined in your source code. In order to do this, Firebase CLI runs a small tool (let's call this a harness) that runs and inspects the functions defined in your source code.

Today, there are 2 different ways this process can work:

1. Legacy Firebase CLI includes a small harness that it uses to discover functions defined in your code.
2. Functions SDK includes the harness so it can bootstrap itself and advertise the functions defined in your code.

We made the shift to prefer (2) if possible as a way to have a language-agnostic process for discovering functions defined in the source code.

Unfortunately, we didn't anticipate how difficult it would be to locate the Firebase Functions SDK package in your source code. We really only considered the simple case where node_modules directory exists inside your functions package. We didn't foresee complication with using more advanced dependency/build tools like pnpm, nx, and other monorepo utilities.

For now, I'm looking to make a quick patch to address the scenarios raised in here, but I hope we can spend some time in the future to make the overall process more reliable. Stay tuned.