Error getting documents: Error: 9 FAILED_PRECONDITION when using collectionGroup

[dumitruPuggle]

Environment details

• OS: 13.2.1 (22D68)
• Node.js version: v18.15.0
• npm version: v9.5.0
• @google-cloud/firestore version: I was told to come to this repo to get the responses quickly, although I'm using firestore-admin 11.9.0

Steps to reproduce

1. Use any collectionGroup and call get function

 db.collectionGroup("textTemplates")
  .where("id", "==", templateId)
  .get()

3. You'll receive an error:

Google API requested!
 - URL: "https://oauth2.googleapis.com/token"
 - Be careful, this may be a production service.
>  Error getting documents:  Error: 9 FAILED_PRECONDITION: 
>      at callErrorFromStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
>      at Object.onReceiveStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client.js:357:73)
>      at Object.onReceiveStatus (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/resolving-call.js:94:78
>      at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
>  for call at
>      at ServiceClientImpl.makeServerStreamRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/client.js:340:32)
>      at ServiceClientImpl.<anonymous> (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/v1/firestore_client.js:227:29
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streamingApiCaller.js:38:28
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
>      at Object.request (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streaming.js:130:40)
>      at makeRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/retry-request/index.js:141:28)
>      at retryRequest (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/retry-request/index.js:109:5)
>      at StreamProxy.setStream (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streaming.js:121:37)
>      at StreamingApiCaller.call (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/node_modules/google-gax/build/src/streamingCalls/streamingApiCaller.js:54:16)
>  Caused by: Error
>      at Query._get (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/reference.js:1717:23)
>      at Query.get (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/@google-cloud/firestore/build/src/reference.js:1705:21)
>      at startTextSequencesHandler (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/lib/functions/callable/start-text-sequences.js:25:10)
>      at fixedLen (/Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/firebase-functions/lib/v2/providers/https.js:118:31)
>      at /Users/dumitrucucu/Desktop/VRSEN/admagic-ai/back/functions/node_modules/firebase-functions/lib/common/providers/https.js:467:32
>      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
>      at async runFunction (/Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:9)
>      at async runHTTPS (/Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:531:5)
>      at async /Users/dumitrucucu/.nvm/versions/node/v18.15.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:693:21 {
>    code: 9,
>    details: '',
>    metadata: Metadata { internalRepr: Map(0) {}, options: {} }
>  }

I created the necessary indexes on Firestore Console:

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

[tom-andersen]

I suspect you are using OAuth 2.0.

See: https://firebase.google.com/docs/admin/setup

Note: OAuth 2.0 refresh tokens are not supported for connecting to Cloud Firestore.

[dumitruPuggle]

I found the issue. All I had to do is to use orderBy, and after that I received an index link for firestore console.

[diginikkari]

I think that the main issue here is that the FAILED_PRECONDITION error is not including create query index -link.

[leonardopn]

My firebase-admin: ^11.10.1

Same problem running this code snippet in a deployed code:

static async getAllByIdProduct(id: string) {
	const query = await firestore
		.collectionGroup("modelos")
		.where("idsDeProduto", "array-contains", id)
		.get();

	return query.docs.map(doc => ({
		data: doc.data() as RoteiroModelDTO,
		ref: doc.ref as DocumentReference<RoteiroModelDTO>,
	}));
}

Error in GCP:

It really seems that the link to create the indexes should come, but it's just showing a generic error

[dconeybe]

Thank you for this additional information. @tom-andersen will follow up early next week.

[tom-andersen]

Update:

@leonardopn I verified the server logs, and the error message describing the problem along with link to create index was provided. But somewhere from the time that response is sent to cloud function and the output being logged, the error message is being stripped out. I see no reason for why the SDK should do this, so I wonder if there is something in the logging setup that causes this.

If you run the query locally on your development machine, I imagine the error with link should appear.

I will see if I can reproduce error with cloud function.

[dumitruPuggle]

Hi guys

Was this issue resolved? I'm asking because I'm sill getting this error in production.

[tom-andersen]

Hi @dumitruPuggle

We still don't know why the text is stripped from the error. However, the resolution remains the same. Create the required composite index. For now you will need to run the code locally (or not within cloud function), and then there will be a helpful link as part of error.

[ivanmartinvalle]

Running into this also. As an alternative to running the query locally, I was able to get the index creation link by using the GCP Console with the Query Builder. But even if the link wasn't generated via that method, it does allow you to quickly test if a particular query would be supported, and then manually go through the process of adding the index in the GCP Console.

[dumitruPuggle]

Hey guys,

I found an alternative solution to fix this problem. Here are the steps:

1. Go to Firestore

2. Click on Query builder tab

3. Replicate the same query through UI:

That's it. You get the index link in console.

[astanb]

This is absolutely ridiculous, you can't warn us when developing and you 'don't know why the text is stripped from the error', so we can't even set up a proper alert in case a missing index gets through to production. Come on Google, do better.

[jelling]

I was referred here by GCP support - which we pay for - so it's extra frustrating to see that a repeatable issue affecting basic functionality is known but receiving no attention. GCP support also referred me to this likely related issue that has been open since 2022.

It is impossible for me to believe that with all the smart people I know at Google that no one can figure out why an error message is getting truncated. So when can we expect a resolution for this?

I know no one the support team is going to lose sleep if we leave Firebase, but we have had so many similar Firebase support experiences - long open issues with basic functionality - that it's increasingly hard to believe that GCP sees Firebase as a priority.

[kossnocorp]

I believe that the issue should receive higher priority. It seems to have slipped through the cracks, and the team isn't really aware of the problem.

Not being able to receive the index link makes the collection group's DX hellish. Let's alone say we have to deploy to production and hope to catch the errors before users do, but we also have to guess which query is failing.

@MarkDuckworth forgive me for pinging you. I can see your commits in the main branch, so hopefully, you can access the actual issue priority.

[tom-andersen]

Hi @kossnocorp,

I am trying to move this forward, since this has somewhat fallen between teams. So here are some updates:

1. Logging is not the cause. The text message is stripped before exception is emitted as part of Firebase Function.
2. I have produced a reproduction and shared it with Firebase Functions. They will investigate whether this happens as part of Firebase Function or further back in the system under a Cloud Function. Someone from FIrebase Functions will investigate further.

When we know more, I will update this issue.

[jelling]

I’m fairly certain this is not related to Firebase functions as I am having the problem while using Next.JS to query Firestore.

On Tue, Apr 30, 2024 at 12:30 PM Tom Andersen @.***> wrote:

Hi @kossnocorp https://github.com/kossnocorp,

I am trying to move this forward, since this has somewhat fallen between teams. So here are some updates:

1. Logging is not the cause. The text message is stripped before exception is emitted as part of Firebase Function.
2. I have produced a reproduction and shared it with Firebase Functions. They will investigate whether this happens as part of Firebase Function or further back in the system under a Cloud Function. Someone from FIrebase Functions will investigate further.

When we know more, I will update this issue.

— Reply to this email directly, view it on GitHub https://github.com/googleapis/nodejs-firestore/issues/1866#issuecomment-2086146218, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOMZ7SGJKLDDBK5SJBMESDY77IMXAVCNFSM6AAAAAA2XF5LCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBWGE2DMMRRHA . You are receiving this because you commented.Message ID: @.***>

[tom-andersen]

This problem sometimes occurs in NodeJS 18 and 20, but not 16 from our observation.

For Cloud Functions and Firebase Functions, reducing the engine to NodeJS version 16 removes the problem.

[sirusbaladi]

orderBy

this fixed it for me.

[leblancmeneses]

I've been a Firebase user since 2016. This error did not include the link to create the missing index, making it difficult to manually create this collection group index.

This worked in the emulator, but in production, it failed because the emulator doesn't use indexes. Not having the link resulted in an unnecessary time sink. I'm lucky to have known what query I added and prior projects I could take the collection group index out to modify for this new application I was working on.

{
  "indexes": [],
  "fieldOverrides": [
    {
      "collectionGroup": "shortredirectlink_shortlink_campaigns_cg",
      "fieldPath": "short_code",
      "indexes": [
        {
          "order": "ASCENDING",
          "queryScope": "COLLECTION"
        },
        {
          "order": "DESCENDING",
          "queryScope": "COLLECTION"
        },
        {
          "arrayConfig": "CONTAINS",
          "queryScope": "COLLECTION"
        },
        {
          "order": "ASCENDING",
          "queryScope": "COLLECTION_GROUP"
        }
      ]
    }]
}

[dalenguyen]

Agree. I'm having the same issue today and the error log is not helpful.

[ohprettyhak]

I am still experiencing this issue today. It is not possible to create the index through the logs.