extensions icon indicating copy to clipboard operation
extensions copied to clipboard

๐Ÿ› [firestore-bigquery-export] Cannot install a new instance after deleting the default service account more than 30 days ago.

Open RyosukeNishi opened this issue 8 months ago โ€ข 21 comments

[READ] Step 1: Are you in the right place?

Issues filed here should be about bugs for a specific extension in this repository. If you have a general question, need help debugging, or fall into some other category use one of these other channels:

  • For general technical questions, post a question on StackOverflow with the firebase tag.
  • For general Firebase discussion, use the firebase-talk google group.
  • To file a bug against the Firebase Extensions platform, or for an issue affecting multiple extensions, please reach out to Firebase support directly.

[REQUIRED] Step 2: Describe your configuration

  • Extension name: firestore-bigquery-export (storage-resize-images, firestore-send-email, etc)
  • Extension version: 0.2.0
  • Configuration values (redact info where appropriate):
    • omited

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:

  1. Delete the default service account.
  2. Try to install the extention.
Expected result

The installation succeeds.

Actual result
; RESOURCE_ERROR at /deployments/firebase-ext-firestore-bigquery-export-tife/resources/fsexportbigquery: {"ResourceType":"gcp-types/cloudfunctions-v2beta:projects.locations.functions","ResourceErrorCode":"404","ResourceErrorMessage":{"code":404,"message":"Service account projects/-/serviceAccounts/(redacted)[email protected] was not found.","status":"NOT_FOUND","statusMessage":"Not Found","requestPath":"https://cloudfunctions.googleapis.com/v2beta/projects/(redacted)/locations/us-central1/functions","httpMethod":"POST"}}

I suspect that, during the installation process, a GCE instance tries to create a Cloud Function instance with the default service account, such as ***[email protected]. If you happen to have removed the default account more than 30 days ago, however, you have no means to install the extension any more because you cannot restore the account nor can you choose which service account to use for the installation.

RyosukeNishi avatar Apr 22 '25 07:04 RyosukeNishi

Hi! I think if you manually recreate that service account it should just start working again?

cabljac avatar Apr 28 '25 08:04 cabljac

Thank you for your suggestion. I suspect that you cannot manually create a service account with the domain @developer.gserviceaccount.com. When you create one manually, the domain will be @PROJECT_ID.iam.gserviceaccount.com.

ref. https://cloud.google.com/iam/docs/service-accounts-create

RyosukeNishi avatar Apr 28 '25 08:04 RyosukeNishi

ah I see, yes it's not possible to recreate it.

I'll raise this with the team and see if we can come up with a solution for you

cabljac avatar Apr 29 '25 13:04 cabljac

Hi, I think the best thing is to raise a customer support ticket about this, they will be able to access your project and assist further.

cabljac avatar May 02 '25 10:05 cabljac

+1

felansu avatar May 06 '25 12:05 felansu

Im in the same situation of @RyosukeNishi Default service account deleted many years ago I contact with google cloud support and they tell for create another service account - but that don't resolves the problem because is expected that format ${projectNumber}[email protected] and we can only create account like blablabla@${project-name}.iam.gserviceaccount.com

felansu avatar May 06 '25 12:05 felansu

Thanks @felansu ill raise this again with the team to find out what we/you should do.

Seems like a limitation of the platform at the moment

cabljac avatar May 06 '25 12:05 cabljac

@cabljac thanks โ€” it should be possible to specify the service account I want to use.

We need this urgently. When I tried uploading all extensions using a specified service account by modifying the code, the extension was uploaded, but I got an error in the fsexportbigquery function:

; RESOURCE_ERROR at /deployments/firebase-ext-activity-log/resources/fsexportbigquery: {"ResourceType":"gcp-types/cloudfunctions-v2beta:projects.locations.functions","ResourceErrorCode":"404","ResourceErrorMessage":{"code":404,"message":"Service account projects/-/serviceAccounts/[email protected] was not found.","status":"NOT_FOUND","statusMessage":"Not Found","requestPath":"https://cloudfunctions.googleapis.com/v2beta/projects/projectId/locations/us-central1/functions","httpMethod":"POST"}}

So basically we can't have more new data in bigQuery because i can't install plugins

felansu avatar May 06 '25 12:05 felansu

I tried this gcloud alpha command to set the default account, but it didn't work. Having read this document, I suppose the problem could be solved if I can specify which account to use for the installation.

RyosukeNishi avatar May 08 '25 07:05 RyosukeNishi

@cabljac

I suppose the problem could be solved if I can specify which service account to use for the installation.

Is this possible if I use CLI?

RyosukeNishi avatar May 08 '25 07:05 RyosukeNishi

@RyosukeNishi i executed successfully but that don't resolve the problem:

gcloud compute project-info set-default-service-account \
--project=idProject
[email protected]

I had a meeting today with firebase and google compute developers, and explain the problem - i explain 2 possible solutions from my point of view:

  1. The most easy, direct, fast and awesome way to resolve: allow to create the default account, think with me, if the code of firebase-tools is using hardcoded account, why exist the possibility for remove that service account? don't make sense, so - allow restore service account (Although it was eliminated 40 years ago) and don't allow more remove that service account

  2. Or - allow pass service account by parameter and considerate as default

The team said will respond in 2 works days (next Tuesday)

felansu avatar May 09 '25 13:05 felansu

@felansu Thank you for sharing. I'm looking forward to seeing their response.

RyosukeNishi avatar May 12 '25 02:05 RyosukeNishi

Google Cloud Support respond me:

I would like to inform you that I have thoroughly reviewed your case and discussed it with our Internal Specialist team. As per their update, I have forwarded your query to the Product Engineering team regarding is there a way to make the value for the Firebase extension configurable instead of hardcoded? for deeper insights. I will provide you a comprehensive update, with potential solutions, within three business days but no later than i.e. May 15th, 2025.

felansu avatar May 14 '25 11:05 felansu

FYI: Though you may have already heard, the team is preparing a PR which removes the hard-coded part. https://github.com/firebase/firebase-tools/pull/8566/

RyosukeNishi avatar May 15 '25 02:05 RyosukeNishi

ah excellent news!

cabljac avatar May 15 '25 06:05 cabljac

How is going guys?

felansu avatar Jun 09 '25 11:06 felansu

Hi @felansu, I have no news so far on the backend fix for that workaround. I will bring this up with the extensions team tomorrow and provide updates as soon as they're available to me.

cabljac avatar Jun 09 '25 12:06 cabljac

Hi, just checking if there are any updates on the issue. Thanks!

hernandoKoggi avatar Sep 11 '25 19:09 hernandoKoggi

Hi @hernandoKoggi, thanks for checking in.

Unfortunately, there are no meaningful updates from Google on this issue. Iโ€™ve escalated it twice already, and the latest response from Google Support (case #62018275) confirmed that the problem is caused by the Firestore โ†’ BigQuery streaming extension being hardcoded to use the default service account. Unlike other Google Cloud services (e.g., Cloud Run) where you can simply pass a service account parameter, this extension does not allow specifying an alternate account.

The really frustrating part is that Googleโ€™s own support acknowledged this limitation and said that the ability to restore or override the default service account will only be available in the second half of 2026. That basically leaves projects like ours stuck with a hardcoded, non-configurable behavior in a production-critical integration for almost a year and a half.

It feels quite amateur for such a deep Google Cloud component to miss something as fundamental as allowing a custom service account, especially given that every other major GCP service supports it. For now, the only workaround they suggested was to pull the extension source and deploy it manually as Cloud Functions, which is not ideal.

So in short: no resolution yet, and the official fix is scheduled very far out.

felansu avatar Sep 11 '25 20:09 felansu

Hi all, I will re-escalate this and provide an update.

cabljac avatar Nov 21 '25 14:11 cabljac

@cabljac Thanks for the update!

Just so you know, the whole team is all over this โ€” weโ€™ve literally made t-shirts with the activity number on them and weโ€™re all wearing them until this gets fixed. ๐Ÿ˜„๐Ÿ‘•

felansu avatar Nov 22 '25 14:11 felansu