Deploying firestore.indexes.json breaks singleField exemption index

[skillitzimberg]

[REQUIRED] Environment info

firebase-tools: 13.11.2

Platform: masOS

[REQUIRED] Test case

A scheduled function fails with Error: 9 FAILED_PRECONDITION.

Creating a collectionGroup exemption index in the Firebase Hosting console fixes the problem. However subsequent hosting deploys break the exemption and scheduled function.

Deploying firebase.indexes.json (firebase deploy --only firestore) also breaks exemption index created in the Firebase console, even when the firebase.indexes.json is the same as the output of firebase firestore:indexes. The exemption fails with Error: 9 FAILED_PRECONDITION

Deleting the exemption created in the console & deploying the firebase.indexes.json, creates the expected exemption, but it fails with Error: 9 FAILED_PRECONDITION.

[REQUIRED] Steps to reproduce

Create a collection that includes a subcollection called "reports" & a scheduled function to remove documents in any collection called "reports".

exports.scheduledFunction = functions.pubsub
  .schedule("every 30 minutes")
  .onRun(async () => {
    return Promise.all([_clearOldDocuments()]);
  });

async function _clearOldDocuments() {
  const date = Date.now() - 600000; // 10 min
  const reports = await admin
    .firestore()
    .collectionGroup("reports")
    .where("generateDate", "<", date)
    .get();
  const batch = admin.firestore().batch();
  reports.forEach((d) => batch.delete(d.ref));
  await batch.commit();
}

[REQUIRED] Expected behavior

The scheduled function runs and deletes the expected documents.

Deploying firebase.indexes.json with the following form creates the index exemption and the scheduled function succeeds:

{
  "indexes": [],
  "fieldOverrides": [
    {
      "collectionGroup": "reports",
      "fieldPath": "generateDate",
      "ttl": false,
      "indexes": [
        {
          "order": "ASCENDING",
          "queryScope": "COLLECTION"
        },
        {
          "order": "DESCENDING",
          "queryScope": "COLLECTION"
        },
        {
          "arrayConfig": "CONTAINS",
          "queryScope": "COLLECTION"
        },
        {
          "order": "DESCENDING",
          "queryScope": "COLLECTION_GROUP"
        }
      ]
    }
  ]
}

[REQUIRED] Actual behavior

The scheduled function fails with Error: 9 FAILED_PRECONDITION.

Creating an index collectionGroup exemption in the console fixes the scheduled function run. However, subsequent hosting deployments cause the function to break.

The only way to fix the function is to deploy an empty firebase.indexes.json ({}), deleting the index exemption in the console, and recreating the index in the console. Any hosting deployments break the scheduled function.

[aalej]

Hey @skillitzimberg, thanks for reaching out and for providing a detailed report. Let me know if I may have misunderstood anything, but from what I can gather, you are encountering an Error: 9 FAILED_PRECONDITION. error in your scheduled function.

It’s likely that indexes for the query you’re trying to run haven't been generated yet, or are being overwritten by a different one. My guess is that it’s the latter since you mentioned it breaks after deployments.

After deploying the firebase.indexes.json you shared, I tried modifying the code snippet you provided to instead use a descending order and no error were raised:

async function _clearOldDocuments() {
   const date = Date.now() - 600000; // 10 min
   const reports = await admin.firestore()
       .collectionGroup("reports")
       .where("generateDate", "<", new Date(date))
       .orderBy("generateDate", "desc")
       .get();
   for (let doc of reports.docs) {
       console.log(doc.data())
   }
   // const batch = admin.firestore().batch();
   // reports.forEach((d) => batch.delete(d.ref));
   // await batch.commit();
}

Since a descending order worked, I’m guessing the query is using an ascending order. That said, you could either update the query in your code to use a descending order, or modify your indexes to include ascending order like so:

firestore.indexes.json

{
 "indexes": [],
 "fieldOverrides": [
   {
     "collectionGroup": "reports",
     "fieldPath": "generateDate",
     "ttl": false,
     "indexes": [
       {
         "order": "ASCENDING",
         "queryScope": "COLLECTION"
       },
       {
         "order": "DESCENDING",
         "queryScope": "COLLECTION"
       },
       {
         "arrayConfig": "CONTAINS",
         "queryScope": "COLLECTION"
       },
       {
         "order": "DESCENDING",
         "queryScope": "COLLECTION_GROUP"
       },
       {
         "order": "ASCENDING",
         "queryScope": "COLLECTION_GROUP"
       }
     ]
   }
 ]
}

Also you mentioned that hosting deploys break the exemption and scheduled function. Could you elaborate on that? When deploying to hosting are you using firebase deploy --only hosting?

[google-oss-bot]

Hey @skillitzimberg. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[google-oss-bot]

Since there haven't been any recent updates here, I am going to close this issue.

@skillitzimberg if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.