firebase
[Stream Firestore to BigQuery] Runtime Status: Processing Description Configuring BigQuery Sync
I am trying to use the extension to Stream Firestore to BigQuery. The initial setup triggered a number of errors which were all permission-related. I then granted the roles that seemed to be required to the service account. It has been a few hours now stuck in runtime status: processing. How much longer should I wait? Please advise.
@cabljac I would very much appreciate it if you can advise here.
Hi @james-wang-careerduck - thanks for raising this, hope i can help.
The extension should grant permissions automatically. Can you provide logs of the errors at all?
If you add a document to the collection specified in params, do you see it get reflected in BigQuery at all?
No, not reflecting in BigQuery at all. The extension installation process never reached completion.
I have working collections and am able to export collections to BigQuery manually.
This morning I saw an update to the extension is available. I proceeded to install the update, then got an error updating
Error Log #1
ERROR 2025-04-22T19:16:55.389903356Z Validation failed for trigger projects/decisive-fold-436618-h2/locations/us-central1/triggers/ext-firestore-bigquery-export-fsexportbigquery-070677: The request was invalid: generic::invalid_argument: Database '(default)' does not exist in region 'us-central1'. Did you mean region 'nam5'?. See 'gcloud firestore https://cloud.google.com/sdk/gcloud/reference/firestore/databases/list to list all Firestore databases under the project.
Error Log #2
ERROR: failed to create image cache: accessing cache image "us-central1-docker.pkg.dev/decisive-fold-436618-h2/gcf-artifacts/ext--firestore--bigquery--export--setup_big_query_sync/cache:latest": connect to repo store "us-central1-docker.pkg.dev/decisive-fold-436618-h2/gcf-artifacts/ext--firestore--bigquery--export--setup_big_query_sync/cache:latest": GET https://us-central1-docker.pkg.dev/v2/token?scope=repository%3Adecisive-fold-436618-h2%2Fgcf-artifacts%2Fext--firestore--bigquery--export--setup_big_query_sync%2Fcache%3Apull&service=: DENIED: Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/decisive-fold-436618-h2/locations/us-central1/repositories/gcf-artifacts" (or it may not exist)
Hi, the first error seems like a misconfiguration, have you made sure you're pointing the extension to the correct database region for your firestore? It looks like your firestore instance is on nam5 (multi region US)
ERROR 2025-04-22T19:16:55.389903356Z Validation failed for trigger projects/decisive-fold-436618-h2/locations/us-central1/triggers/ext-firestore-bigquery-export-fsexportbigquery-070677: The request was invalid: generic::invalid_argument: Database '(default)' does not exist in region 'us-central1'. Did you mean region 'nam5'?. See 'gcloud firestore https://cloud.google.com/sdk/gcloud/reference/firestore/databases/list to list all Firestore databases under the project.
Closing as stale. If the issue persists could you open a fresh issue with the information requested (what region you're configuring for the firestore instance etc)