firebase-tools icon indicating copy to clipboard operation
firebase-tools copied to clipboard

Published 20 hours ago • verified publisher icon firebase

Flaky Cloud Function v2 deploy - `There was an error retrieving the Firestore database`

Open kaueDM opened this issue 1 year ago • 4 comments

[REQUIRED] Environment info

firebase-tools: 13.15.2

Platform: Windows

[REQUIRED] Test case

Try to deploy Cloud Functions V2 (mix between callables, https and Firestore triggers)

firebase deploy --only functions

Currently I have more than 70 cloud functions in my project. Doesn't matter if I'm trying to deploy 1 or all of them, 80% of the time I get the message:

Failed to make request to https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default)
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.

After many retries, the deploy works. So we can exclude any code issues in the Cloud Functions. This has been happening in the last weeks and lately it has been impacting my productivity a lot. Sometimes I have to retry for 20 minutes to get a working deploy.

I also traced and pinged the IP reported on the debug and it is working normally.

[REQUIRED] Steps to reproduce

See above

I also tried uninstall & install firebase-tools, have installed every single update and reauthenticated multiple times, without any change.

[REQUIRED] Expected behavior

Successful deploy or at least an useful error message.

[REQUIRED] Actual behavior

Relevant part:

[debug] [2024-08-16T18:54:10.636Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default) failed, reason: connect ETIMEDOUT <IP ADDRESS>:443
[info] There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[debug] [2024-08-16T18:54:10.637Z] FetchError: request to https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default) failed, reason: connect ETIMEDOUT <IP ADDRESS>:443
    at ClientRequest.<anonymous> (C:\Users\<MY USER>\AppData\Roaming\npm\node_modules\firebase-tools\node_modules\node-fetch\lib\index.js:1501:11)
    at ClientRequest.emit (node:events:517:28)
    at ClientRequest.emit (node:domain:489:12)
    at TLSSocket.socketErrorListener (node:_http_client:501:9)
    at TLSSocket.emit (node:events:517:28)
    at TLSSocket.emit (node:domain:489:12)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
[error] 
[error] Error: Failed to make request to https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default)
[debug] [2024-08-16T18:54:10.638Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default) failed, reason: connect ETIMEDOUT <IP ADDRESS>:443
[info] There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[debug] [2024-08-16T18:54:10.639Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/<MY PROJECT>/databases/(default) failed, reason: connect ETIMEDOUT <IP ADDRESS>:443
[info] There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.

kaueDM avatar Aug 16 '24 19:08 kaueDM

Hey @kaueDM, sorry to hear you’re encountering this issue and thanks for reaching out to us. Based on the ETIMEDOUT error message, it looks like this could be a network related issue. Would you be able to try connecting to a different network to see if the behavior would change?

Also, I’m currently not able to reproduce the issue. Could you let us know how many Firestore trigger functions are in your codebase? I understand that the behavior is inconsistent, having a similar codebase to the one you have might help us with reproducing the issue.

aalej avatar Aug 19 '24 15:08 aalej

Hey @aalej, thanks for the reply. I've already tried a different connection but the inconsistency persists. As I said in the original report, the connection seems OK since Test-NetConnection (powershell) returns a positive feedback. My current project contains 75 Cloud Functions V2 from multiple types: Firestore Triggers, HTTPS, Callables and Tasks. I started noticing the issue around ~50 CFs, but not that flaky as seem on the last weeks.

kaueDM avatar Aug 19 '24 15:08 kaueDM

I discussed this with our team, currently it’s still pretty difficult to determine what specifically is causing the ETIMEDOUT error. I understand that you were able to ping the IP without any issues, but from what we can gather, generally ETIMEDOUT is usually caused by network related issues like intermittent connection. Though it may be possible that the Firebase CLI might be using a misconfigured proxy setting.

While we try to investigate what could be causing this, if you have any proxy setup, could you try temporarily disabling those to see if there would be any changes in behavior during deployment?

aalej avatar Aug 21 '24 13:08 aalej

I am running into a similar problem. It's almost like it's making to many requests. Some succeed but the rest all fail.

[2024-10-17T06:48:46.443Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.444Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.444Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.445Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.445Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.445Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.446Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.446Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.446Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.446Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.447Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.447Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.447Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.447Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.448Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.448Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.448Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.448Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.448Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.449Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.449Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.449Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.449Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.450Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.450Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.450Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.450Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.451Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.451Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.451Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.451Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.451Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.452Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.452Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.452Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.452Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.453Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.453Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:46.453Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) [none]
[2024-10-17T06:48:47.804Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:47.804Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:47.816220Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"II3H0qDplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.807Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.808Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.819651Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IPPTj6HplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.888Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.888Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.896209Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IJC9lKHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.890Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.890Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.901062Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IILklKHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.891Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.891Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.902043Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IKvslKHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.891Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.891Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.905714Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IKKIlaHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.892Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.892Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.901095Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IJHjlKHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.893Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.893Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.906049Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IKeGlaHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.899Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.899Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.911932Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IKm3laHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.901Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.901Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.908256Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IIOclaHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.901Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.901Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.911441Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IIy1laHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.901Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.901Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.915094Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"INPPlaHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.902Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.902Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.908814Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IK2hlaHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.903Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.903Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.911443Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IIS1laHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.904Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.904Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.919159Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IKPvlaHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.911Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.912Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.922909Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IJWNlqHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.920Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.920Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.931720Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IPPBlqHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.922Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.922Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.936705Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IIT6lqHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.923Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.923Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.936916Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IMr8lqHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.926Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.926Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.934362Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IIHnlqHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:48.935Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:48.935Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:48.946695Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IKHAl6HplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:49.902Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:49.902Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:49.915509Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"INjX0qHplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:50.884Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:50.885Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:50.894731Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"II25jqLplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:50.885Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:50.885Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:50.894049Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"INuqjqLplIkDMMWGmanuzoQD"}
[2024-10-17T06:48:50.898Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) 200
[2024-10-17T06:48:50.898Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) {"name":"projects/gproject-id/databases/(default)","uid":"12d565f3-60b7-43ca-aa88-f85929523600","createTime":"2021-01-09T09:31:42.968Z","updateTime":"2022-01-31T23:21:24.125142Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-10-17T05:48:50.912125Z","appEngineIntegrationMode":"ENABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IPa4j6LplIkDMMWGmanuzoQD"}
[2024-10-17T06:50:01.461Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.467Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.468Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.468Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.468Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.468Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.469Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.469Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.469Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.469Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.470Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.474Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.474Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.475Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-10-17T06:50:01.859Z] FetchError: request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default) failed, reason: connect ETIMEDOUT 192.178.54.170:443
    at ClientRequest.<anonymous> (/opt/homebrew/lib/node_modules/firebase-tools/node_modules/node-fetch/lib/index.js:1501:11)
    at ClientRequest.emit (node:events:519:28)
    at ClientRequest.emit (node:domain:488:12)
    at TLSSocket.socketErrorListener (node:_http_client:500:9)
    at TLSSocket.emit (node:events:519:28)
    at TLSSocket.emit (node:domain:488:12)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Error: Failed to make request to https://firestore.googleapis.com/v1/projects/gproject-id/databases/(default)

Wackymax avatar Oct 17 '24 06:10 Wackymax

Any update on this? I'm not using any proxy, keeping up with the latest releases and the problem persists.

Currently using 83 cloud functions.

kaueDM avatar Dec 11 '24 13:12 kaueDM

I'm facing the same issue. The logs below are from an attempt at deploying 1 v2 function. They worked fine yesterday. I've checked IAM credentials and they look like they should be sufficient. Some GETs are also returning 200s which is confusing. However, there is a lot of repetition which I don't know whether it's normal or not. The database does exist and location hasn't changed since they last time this deployed smoothly.

[2024-12-11T16:43:58.544Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.545Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.546Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] Checked if tokens are valid: true, expires at: 1733938117494
[2024-12-11T16:43:58.547Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.548Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.549Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.549Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.550Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.550Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.551Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.551Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.552Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.552Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.553Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.553Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:58.553Z] >>> [apiv2][query] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) [none]
[2024-12-11T16:43:59.568Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) failed, reason:
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-12-11T16:43:59.571Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) failed, reason:
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-12-11T16:43:59.572Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) failed, reason:
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-12-11T16:43:59.572Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) failed, reason:
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-12-11T16:43:59.573Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) failed, reason:
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-12-11T16:43:59.574Z] *** [apiv2] error from fetch(https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default), {"headers":{},"method":"GET"}): FetchError: request to https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) failed, reason:
There was an error retrieving the Firestore database. Currently, the database id is set to (default), make sure it exists.
[2024-12-11T16:43:59.727Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) 200
[2024-12-11T16:43:59.727Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) {"name":"projects/[REDACTED]/databases/(default)","createTime":"2023-06-23T07:56:17.760793Z","updateTime":"2023-06-23T07:56:17.760793Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-12-11T15:43:59.701673Z","appEngineIntegrationMode":"DISABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"IM7Iye+UoIoDMJ6YltKB34kD"}
[2024-12-11T16:43:59.749Z] FetchError: request to https://firestore.googleapis.com/v1/projects/[REDACTED]/databases/(default) failed, reason:
    at ClientRequest.<anonymous> (C:\Users\MY-PC-LOCAL\AppData\Roaming\nvm\v20.18.0\node_modules\firebase-tools\node_modules\node-fetch\lib\index.js:1501:11)
    at ClientRequest.emit (node:events:519:28)
    at ClientRequest.emit (node:domain:488:12)
    at emitErrorEvent (node:_http_client:101:11)
    at TLSSocket.socketErrorListener (node:_http_client:504:5)
    at TLSSocket.emit (node:events:519:28)
    at TLSSocket.emit (node:domain:488:12)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Error: Failed to make request to https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default)
[2024-12-11T16:43:59.775Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) 200
[2024-12-11T16:43:59.776Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) {"name":"projects/[PROJECT_ID]/databases/(default)","uid":"[UID_REDACTED]","createTime":"2023-06-23T07:56:17.760793Z","updateTime":"2023-06-23T07:56:17.760793Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-12-11T15:43:59.749331Z","appEngineIntegrationMode":"DISABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"[ETAG_REDACTED]"}
[2024-12-11T16:43:59.778Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) 200
[2024-12-11T16:43:59.779Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) {"name":"projects/[PROJECT_ID]/databases/(default)","uid":"[UID_REDACTED]","createTime":"2023-06-23T07:56:17.760793Z","updateTime":"2023-06-23T07:56:17.760793Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-12-11T15:43:59.749523Z","appEngineIntegrationMode":"DISABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"[ETAG_REDACTED]"}
[2024-12-11T16:43:59.779Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) 200
[2024-12-11T16:43:59.779Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) {"name":"projects/[PROJECT_ID]/databases/(default)","uid":"[UID_REDACTED]","createTime":"2023-06-23T07:56:17.760793Z","updateTime":"2023-06-23T07:56:17.760793Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-12-11T15:43:59.750245Z","appEngineIntegrationMode":"DISABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"[ETAG_REDACTED]"}
[2024-12-11T16:43:59.779Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) 200
[2024-12-11T16:43:59.780Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) {"name":"projects/[PROJECT_ID]/databases/(default)","uid":"[UID_REDACTED]","createTime":"2023-06-23T07:56:17.760793Z","updateTime":"2023-06-23T07:56:17.760793Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-12-11T15:43:59.751983Z","appEngineIntegrationMode":"DISABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"[ETAG_REDACTED]"}
[2024-12-11T16:43:59.781Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) 200
[2024-12-11T16:43:59.781Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) {"name":"projects/[PROJECT_ID]/databases/(default)","uid":"[UID_REDACTED]","createTime":"2023-06-23T07:56:17.760793Z","updateTime":"2023-06-23T07:56:17.760793Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-12-11T15:43:59.751786Z","appEngineIntegrationMode":"DISABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"[ETAG_REDACTED]"}
[2024-12-11T16:43:59.785Z] <<< [apiv2][status] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) 200
[2024-12-11T16:43:59.785Z] <<< [apiv2][body] GET https://firestore.googleapis.com/v1/projects/[PROJECT_ID]/databases/(default) {"name":"projects/[PROJECT_ID]/databases/(default)","uid":"[UID_REDACTED]","createTime":"2023-06-23T07:56:17.760793Z","updateTime":"2023-06-23T07:56:17.760793Z","locationId":"eur3","type":"FIRESTORE_NATIVE","concurrencyMode":"PESSIMISTIC","versionRetentionPeriod":"3600s","earliestVersionTime":"2024-12-11T15:43:59.755049Z","appEngineIntegrationMode":"DISABLED","keyPrefix":"e","pointInTimeRecoveryEnablement":"POINT_IN_TIME_RECOVERY_DISABLED","deleteProtectionState":"DELETE_PROTECTION_DISABLED","etag":"[ETAG_REDACTED]"}

Any thoughts or suggestions warmly welcomed

joeframpton avatar Dec 11 '24 17:12 joeframpton

Since my last message, I made no changes (to the code or to the environment) and it deployed smoothly.

joeframpton avatar Dec 12 '24 09:12 joeframpton

The issue has returned today. I did some digging and added additional logs and got the following (from within firebase-tools:

 FirebaseError: Failed to make request to https://firestore.googleapis.com/v1/projects/<REDACTED>/databases/(default)
    at RetryOperation._fn (C:\Users\MY-PC-LOCAL\AppData\Roaming\nvm\v20.18.1\node_modules\firebase-tools\lib\apiv2.js:261:31)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  children: [],
  context: undefined,
  exit: 1,
  original: FetchError: request to https://firestore.googleapis.com/v1/projects/<REDACTED>/databases/(default) failed, reason:  with code undefined
      at ClientRequest.<anonymous> (C:\Users\MY-PC-LOCAL\AppData\Roaming\nvm\v20.18.1\node_modules\firebase-tools\node_modules\node-fetch\lib\index.js:1503:11)
      at ClientRequest.emit (node:events:518:28)
      at ClientRequest.emit (node:domain:489:12)
      at emitErrorEvent (node:_http_client:101:11)
      at TLSSocket.socketErrorListener (node:_http_client:504:5)
      at TLSSocket.emit (node:events:518:28)
      at TLSSocket.emit (node:domain:489:12)
      at emitErrorNT (node:internal/streams/destroy:169:8)
      at emitErrorCloseNT (node:internal/streams/destroy:128:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
    type: 'system',
    errno: 'ETIMEDOUT',
    code: 'ETIMEDOUT'
  },
  status: 500
}

Similar to @kaueDM, I've pinged the IP associated with the request and the result was:

 ping 142.250.200.10

Pinging 142.250.200.10 with 32 bytes of data:
Reply from 142.250.200.10: bytes=32 time=28ms TTL=115
Reply from 142.250.200.10: bytes=32 time=31ms TTL=115
Reply from 142.250.200.10: bytes=32 time=18ms TTL=115
Reply from 142.250.200.10: bytes=32 time=28ms TTL=115

Ping statistics for 142.250.200.10:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 18ms, Maximum = 31ms, Average = 26ms

Additionally, this timeout message seems to arrive very quickly. Coupled with the 500, I wonder if there's something else going on?

joeframpton avatar Dec 16 '24 16:12 joeframpton

Since this seems to only happen when deploying many Firestore functions at once, we might be able to address this by caching the reponse instead of repeating the same call over and over.

joehan avatar Dec 17 '24 18:12 joehan

@joehan This happens when deploying a single function though too.

It always seems to happen on particular network - deploys fine when I hotspot. So I wonder if the ISP that's blocking something, or their supplied router.

joeframpton avatar Jan 02 '25 11:01 joeframpton

It's almost certainly a networking/proxy error. I added additional logs and found that alongside the ETIMEDOUT error, there is a ENETUNREACH error. I've also noticed that this error only happens for a particular device on a particular network

  • On a different network (e.g. office wifi or mobile hotspot) the deployment happens smoothly - so presumably rules out user/IAM/device configuration issue
  • On a different device on the same network the deployment happens smoothly - so presumably rules out ISP specific or local network issues
  • The same issue occurs regardless of project selected

So I suspect it's something to do with how something is cached or a parameter is configured? I've tried:

  • Updating npm to 11.1.0
  • Uninstalling and reinstalling firebase-tools (v13.30.0)
  • Changing DNS
  • I don't have proxy env params (HTTP_PROXY or HTTPS_PROXY)

I'd welcome any other suggestions you might have. I'd rather not have to switch to mobile hotspot to deploy functions every time 😅

joeframpton avatar Feb 10 '25 16:02 joeframpton

This happened to me today I tried multiple times but the functions weren't deploying with the error of firestore database not exisiting as one user suggested changing a network I opened my vpn (i wasnt running it before) and then deployed the functions and it worked.

arafays avatar Feb 24 '25 17:02 arafays

Hey all, sorry for the difficulties here. We've tried to repro this on our end to no avail, and all signs point to this being a network connectivity issue thats blocking requests to Firestore APIs. If you are running into this issue, the best bet (other than using a different network) is to reach out to Cloud Support for help.

Since there's nothing we can do in this repo to fix this issue, closing it.

joehan avatar Feb 25 '25 18:02 joehan