Version 11.2.2 and higher do not attach secrets to functions upon deploy when using Firebase Functions SDK <v.3.20.0

[jacobsandlund]

[REQUIRED] Environment info

firebase-tools: 11.2.2,11.3.0 or 11.4.0 (has bug)

firebase-tools: 10.9.2 and 11.1.0 succeeds (no bug)

Platform: macOS

[REQUIRED] Test case

Have a function with runWith secrets:

export const playground = functions
  .runWith({
    secrets: [
      'TEST_SECRET',
    ],
  })
  .https.onRequest(app());

[REQUIRED] Steps to reproduce

Deploy the function:

firebase deploy --project projectId --config firebase.json --only functions:playground

[REQUIRED] Expected behavior

The secret should be attached to the function, as it is in 10.9.2 and 11.1.0.

[REQUIRED] Actual behavior

There is no log about attaching the TEST_SECRET in the buggy versions when deploying, whereas 10.9.2 and 11.1.0 do attach the secret. Inside the function, inspecting process.env.TEST_SECRET shows undefined in the buggy versions.

[taeold]

Hi @jacobsandlund - apologies for the issue you are seeing.

Thank you for sharing your experiments. You are right - we shipped a bug.

While we work on getting a hotfix out, I suggest you try updating the firebase-functions to the latest version. It seems like the bug only exists on older versions of the functions SDK.

[Schmale97]

I am on firebase-functions version 3.22.0 and firebase-tools versions 11.4.2 and my secrets aren't being picked up when I emulate the cloud functions not will my cloud function deploy, is this the same bug?

[taeold]

@Schmale97 Do you mind sharing a quick example of your setup?

I tried to reproduce your issue using the latest Firebase CLI (11.4.2) and Functions SDK (3.22.0) and running the emulator, but things seemed to work on my setup:

const functions = require("firebase-functions");

exports.helloSecrets = functions.runWith({ secrets: ["MY_SECRET"] }).https.onRequest((request, response) => {
  functions.logger.info("MY_SECRET!", {secret: process.env.MY_SECRET});
  response.send("Hello from Firebase!");
 });

$ firebase emulators:start
i  functions: Beginning execution of "helloSecrets"
>  {"secret":"*****","severity":"INFO","message":"MY_SECRET!"}

[Schmale97]

What other details do you need about my set up? I'm using node 16.16.0

I did the same test as you with a simple onRequest and onCall,

export const helloSecrets = functions.runWith({ secrets: ['MY_SECRET'] }).https.onRequest((request, response) => {
  functions.logger.info('MY_SECRET!', { secret: process.env.MY_SECRET })
  response.send(process.env.MY_SECRET)
})

export const helloSecrets2 = functions.runWith({ secrets: ['MY_SECRET'] }).https.onCall((data, context) => {
  functions.logger.info('MY_SECRET!', { secret: process.env.MY_SECRET })
  return process.env.MY_SECRET
})

firebase-functions: 3.22.0 firebase-tools: 11.1.0 outputted successfully from both functions

{"secret":"***********","severity":"INFO","message":"MY_SECRET!"}

firebase-functions: 3.22.0 firebase-tools: 11.4.2 outputted no secret from both functions

{"severity":"INFO","message":"MY_SECRET!"}

[moldy530]

I'm also experiencing my functions not being created or updated with secrets on firebase-tools 11.5.0 and firebase functions 3.22

[vamshiaruru-virgodesigns]

We are seeing that when we try to set region using environment variables in .env file, and try to use it as functions.region(process.env.REGION), but it is coming up as undefined. But the same variable can be accessed from inside the function body. Is this a bug related to the same bug in this issue? Or is this an expected behaviour?

[taeold]

@vamshiaruru-virgodesigns In that case, it is an expected behavior. Check out https://github.com/firebase/firebase-functions/issues/1084 why that's the case and what we are doing about it.

[dangreaves]

The version of firebase-functions seems to make no difference for me. This seems to be an issue only with firebase-tools.

firebase-functions: 3.22.0 firebase-tools: 11.1.0 Secret is attached correctly, process.env.MY_SECRET contains the secret value.

firebase-functions: 3.22.0 firebase-tools: 11.2.0 (and all versions above that) No secret is attached in the emulator, process.env.MY_SECRET is undefined.

[tpodom]

I'm seeing this as well. Is there anything I can do to help get to the bottom of the issue? I'm using firebase-functions 3.22.0 and firebase-tools 11.7.0.

I didn't notice the issue at first because functions that were previously deployed with secrets still show secrets in Google Cloud. But, deploying new secrets is not working. If I manually add the secret to the function in Google Cloud it seems to stick around after redeploy.

This issue doesn't seem to be a problem with the emulator, it's just when it creates the function in Google Cloud from firebase deploy.

[jowo-io]

I'm seeing this as well. Is there anything I can do to help get to the bottom of the issue? I'm using firebase-functions 3.22.0 and firebase-tools 11.7.0.

I didn't notice the issue at first because functions that were previously deployed with secrets still show secrets in Google Cloud. But, deploying new secrets is not working. If I manually add the secret to the function in Google Cloud it seems to stick around after redeploy.

This issue doesn't seem to be a problem with the emulator, it's just when it creates the function in Google Cloud from firebase deploy.

Same here, experiencing the same issues with v11.7.0 and v3.22.0

[jowo-io]

@Schmale97 Do you mind sharing a quick example of your setup?

I tried to reproduce your issue using the latest Firebase CLI (11.4.2) and Functions SDK (3.22.0) and running the emulator, but things seemed to work on my setup:

const functions = require("firebase-functions");

exports.helloSecrets = functions.runWith({ secrets: ["MY_SECRET"] }).https.onRequest((request, response) => {
  functions.logger.info("MY_SECRET!", {secret: process.env.MY_SECRET});
  response.send("Hello from Firebase!");
 });

$ firebase emulators:start
i  functions: Beginning execution of "helloSecrets"
>  {"secret":"*****","severity":"INFO","message":"MY_SECRET!"}

also tried the versions suggested here on my windows 10 machine, and still no joy. will try again on osx

[tpodom]

@taeold I did some digging between 11.1.0 and 11.8.0. The issue appears to be that there was a change from discoverBackend to discoverBuild. Unfortunately, discoverBuild calls addResourcesToBuild which does not contain the block of code to copy the secrets to secretEnvironmentVariables that addResourcesToBackend does at https://github.com/firebase/firebase-tools/blob/v11.8.0/src/deploy/functions/runtimes/node/parseTriggers.ts#L496-L507

Looks like it was related to this change: https://github.com/firebase/firebase-tools/commit/39bf06e80b79c933a8a3fa5b471962e9d1b02b7c

[jowo-io]

I could not get my project running on osx yesterday (because of some unrelated issues) in order to test it. However, I did try installing v11.0.1 on windows, which is the version directly before v11.1.0, as @tpodom mentioned it might be where the bug was introduced.

And sure enough, the secrets are being deployed when using [email protected]

[eckberg]

I have the same issue, running [email protected] on macOS. No secrets deployed. No errors or issued raised. @tpodom seems to be right on spot on the issue.

As a workaround, I added my secrets manually in the Google Cloud Platform Console following this guide.

[taeold]

@tpodom Thanks for the pointer!

I have a bug out for fixing that error at https://github.com/firebase/firebase-tools/pull/4926

However, I'm a bit worried that this only fixes the bug where function deploy would not attach secrets IF you are using firebase functions SDK below v3.20.0.

I'm still unable to reproduce the issue on my setup 😢 using firebase functions v3.20.0. It would help me tremendously if you can share output of your function deploy, especially the one with the debug flag e.g. firebase deploy --only functions --debug.

[tpodom]

@taeold I've got a debug log attached. But, I think I know why you weren't able to reproduce it.

In my setup, my functions are part of a workspace so the npm list firebase-functions --json=true doesn't return expected format. As a result, the sdkVersion ends up empty and it falls back to parseTriggers which still returns secrets instead of secretEnvironmentVariables.

If I break my package out as a standalone package it parses the firebase-functions version successfully and goes into discovery.detectFromPort which returns secretsEnvironmentVariables and carries forward successfully.

Here's an example of the npm list output that is tripping it up:

{
  "version": "0.1.0",
  "name": "@tpodom/root",
  "dependencies": {
    "@tpodom/functions": {
      "resolved": "file:../../packages/functions",
      "dependencies": {
        "firebase-functions-test": {
          "version": "2.3.0",
          "resolved": "https://registry.npmjs.org/firebase-functions-test/-/firebase-functions-test-2.3.0.tgz",
          "dependencies": {
            "firebase-functions": {
              "version": "3.22.0"
            }
          }
        },
        "firebase-functions": {
          "version": "3.22.0",
          "resolved": "https://registry.npmjs.org/firebase-functions/-/firebase-functions-3.22.0.tgz"
        }
      }
    }
  }
}

broken.debug.log.gz

[vedmakk]

Same issue with:

firebase-functions: 3.21.0 firebase-tools: 11.2.2

I have enabled the Secrets Manager API in google cloud AFTER the function was first deployed… and added secrets later down the road. Although completely deleting the function and re-deploying it didn't help either.

[taeold]

@tpodom Thanks for following up. That gives me some closure on why things weren't working as I'd expect.

Monorepo isn't something we failed to consider when developing in the CLI, and this is yet another reason for us to pay more attention to it. Same story with yarn pnp support.

We have a 4 year old issue - https://github.com/firebase/firebase-tools/issues/653 - if anything you'll hear us from this. Thanks you again for helping us understand the issue better.

[taeold]

@protyze Are you also using something like yarn/npm workspaces the way @tpodom described?

[moldy530]

@protyze Are you also using something like yarn/npm workspaces the way @tpodom described?

Ahh this explains it, we are also using workspaces in our repo where we've been affected by this

[taeold]

Issue is closed, but the fix will only be shipped in next CLI release. I'll try to have our team cut a release tomorrow.

Thanks for your patience.

[vedmakk]

@protyze Are you also using something like yarn/npm workspaces the way @tpodom described?

Ahh, yes, I am also using yarn workspaces! Thanks for the quick response and fix. Looking forward to the next CLI release.

[Thaina]

I still got this warning in 11.16.0 when run in firebase emulator

{"severity":"WARNING","message":"No value found for secret parameter "DIALOGFLOW_KEYFILE". A function can only access a secret if you include the secret in the function's dependency array."}

{"severity":"WARNING","message":"No value found for secret parameter "FACEBOOK_TOKENS". A function can only access a secret if you include the secret in the function's dependency array."}

This is my code

const functionsParams = require('firebase-functions/params');
var FACEBOOK_TOKENS = functionsParams.defineSecret("FACEBOOK_TOKENS");
var DIALOGFLOW_KEYFILE = functionsParams.defineSecret("DIALOGFLOW_KEYFILE");

exports.WebHook = functions.region("asia-east1").runWith({ secrets: [FACEBOOK_TOKENS,DIALOGFLOW_KEYFILE] }).https.onRequest(async (request,response) => {
	console.log(DIALOGFLOW_KEYFILE.value());
	console.log(FACEBOOK_TOKENS.value());

Even I have set my secret properly

It also have weird behaviour that 1 in 10 times it might get the value for unknown reason

[ne1114]

@Thaina I am facing same issue, did you have any luck?

[Thaina]

Not sure, I have not test this flow for such time

[simonhildebrandt]

I seem to be getting this as well...