FirebaseError: Document already exists

[mafergus]

[REQUIRED] Describe your environment

• Operating System version: Multiple (Windows 10, Android 10)
• Browser version: Multiple (Chrome Mobile 91.0.4472, Edge, Samsung Internet 13.0)
• Firebase SDK version: 8.4.3
• Firebase Product: Firestore

We are seeing this error when trying to add a document to a collection. I've scoured Google and Stack-Overflow, gone through the source code and also gprc source code. I can't understand the reason for this error:

firebase.firestore().collection('some-collection').add({ data: 1 })

As far as I can tell, this should never throw a "document exists" error. Is this valid behavior or a Firebase bug?

[schmidt-sebastian]

@mafergus Thanks for reporting this!

add() uses a random client-side generated identifier, so there is a small chance for a collision - but the chance for this is so small that we are willing to say that it should never occur. Are you able to provide debug logs for us, which will then show what identifier causes the collision?

You can enable debug logging by invoking setDebugLevel('debug').

[mafergus]

Yes we're happy to assist in any way possible, you guys are doing great work. Would you mind pointing me to the code that generates this identifier so I can better understand the algorithm? Perhaps we can change our code to mitigate any collisions. We've noticed this specifically during periods of peak activity.

We'll enable the debug logging, I would expect we'll hit this again over the weekend.

[schmidt-sebastian]

The code is here: https://github.com/firebase/firebase-js-sdk/blob/5ad7ff2ae955c297556223e6cb3ad9d4b897f664/packages/firestore/src/util/misc.ts#L28

[google-oss-bot]

Hey @mafergus. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[mafergus]

Hey @schmidt-sebastian, we captured a bunch of examples of this over the weekend in Sentry. As far as I'm aware, Sentry doesn't give a nice way to export breadcrumb logs. The two options I see are either pass you JSON of the breadcrumbs, or temporarily invite you to our Sentry project. Let me know how you'd like to proceed.

[schmidt-sebastian]

It depends how much logging we are talking about here. Please contact me at mrschmidt(at)google.com.

[oveddan]

I'm having this issue too

[schmidt-sebastian]

Are you able to catch and ignore these errors for now? We might need to adjust how these documents are processed internally to address this, which is major change for us. Essentially, what is happening here is that the SDK sends a write and then the connection gets dropped. The backend however executed the write. When we then try to send the write again, the backend tells us that the document already exists as our previous attempt did in fact succeed.

[oveddan]

For me this is a critical part of my application - i use the firestore document as a state record for a user that is created when they open my application, and a cloud function reads that document to determine where to route the user internally.

I've temporarily made a workaround (in the below code, store is my firestore db):

const requestToJoin = async ({
  spaceId,
  userId,
}: {
  spaceId: string;
  userId: string;
}) => {
  try {
    return await store.collection("joinRequests").add({
      spaceId,
      userId,
    });
  } catch (e) {
    console.error(e);
    try {
      console.error("failed joining, retrying");
      return await store.collection("joinRequests").add({
        spaceId,
        userId,
      });
    } catch (e) {
      console.error(e);
      return undefined;
      // notify user they cannot connect to the application
    }
  }
};

This works most of the time. Is there something I can do besides this to prevent this issue? It only started appearing a couple weeks ago.

I'm using firebase-js version 8

[oveddan]

I suppose a workaround is wrapping this add functionality in a cloud function that performs the adding, and the cloud function just returns the added element's id?

[schmidt-sebastian]

It is very likely that you can just ignore "Document already exists" error since it means that our previous attempt to write this document actually succeeded. We cannot do this from the SDK side (yet) as some other client may have interleaved a write and changed the document after the initial write attempt.

[oveddan]

I need the id of the document, so I don't think try catching and swallowing it would work? For now I will wrap it in a cloud fn.

[schmidt-sebastian]

Can you use setDoc(doc(collection), {...}) (v9) or collection.set(collection.doc(), {}) (v8)? This will use auto-generated document IDs but not verify that the document does not exist. Our document IDs are 20 bytes long, so the chance of collisions are low. SetDoc() will succeed if the SDK issues the write twice.

[vandres]

Also seeing the error very frequently in our application. We don't have that much traffic, so I also don't think, an ID collision is very likely.

[vinodgubbala]

Hope, addDoc becomes intelligent enough to mitigate this issue. The amount of errors is quite annoying. I tried setDoc solution. It is working good, but I am not really comfortable with using it on production.

[ginagr]

I am using addDoc and getting this error pretty regularly.

Code:

await addDoc(collection(this.firestore, `audit/${collectionPath}/${id}`), {   ...   });

Error:

[camr1993]

I am also running into this same issue with addDoc

[marshyon]

I am seeing the same issue with addDoc

[lukasvan3l]

I see this issue 4 times in my logs in the past weekend, and a rough guess of the succeeded amount of calls is 30K - 40K. My db structure is /users/{userId}/progress/{progressId} the 30/40K calls are distributed over 5K users, and these were new users, so the /progress collection was empty. So there are only 6 records on average for every user.

4 is a lot of occurrences with this little data, so it might be that there's something else going on than just coincidence in randomness...

[railerbailer]

I'm also experiencing this. I think it's the cause of crashing my entire application, but when authed on same user with incognito, no problemo. Any suggestions?

[MarkDuckworth]

Thanks for continuing to bring this to our attention, your feedback helps us understand how this affects you. We still believe the root cause to be as described in this Feb 14th comment. Additionally, please continue to refer to the workaround in this Feb 17th comment.

[raoneel]

I have figured out why this is happening for me. I am testing some ML libraries that use deterministic seeding (such as seed_everything). In some cases, this can cause the Firestore library to generate the exact same document ID as before. To fix this, I basically just re-seed the random libraries before calling .add(). This fixed the issue for me, hope it helps someone else.

[vojto]

We're getting this error in a transaction that looks like this:

await runTransaction(db, async (tx) => {
	const snapshot = await ref.get()
	if (snapshot.exists()) {
		throw new Error("document exists")
	}

	await ref.set(...)
})

I can't reproduce it, but we have a lot of Sentry logs with it.

The Document already exists error is really confusing here, because we're explicitly ensuring that the document doesn't exist before writing to it. Also using set method should never cause "document already exists", because that method is supposed to write whether it exists or not.

It is very likely that you can just ignore "Document already exists" error since it means that our previous attempt to write this document actually succeeded. We cannot do this from the SDK side (yet) as some other client may have interleaved a write and changed the document after the initial write attempt.

Are you telling me that the change is being saved, and I should just ignore the error? What if you decide to change things in SDK and now the error becomes something I shouldn't ignore?

Is there any chance you could at least rename that error to something like "Document maybe wasn't saved" or something more accurate?'

[vinodgubbala]

@vojto did you recently update your firbase version in your code base. We found this was expected behaviour. Earlier it used to not trigger error but trigger a warning.

So between get and set, if the document state changes the transaction gets failed.

[vojto]

So between get and set, if the document state changes the transaction gets failed.

Yep - the error started showing up only recently. And I've been able to reproduce it by running the code twice at the same time!

I thought in case of concurrent edits, the transaction would be automatically rerun:

A function calling a transaction (transaction function) might run more than once if a concurrent edit affects a document that the transaction reads. (docs)

[MarkDuckworth]

Hi @vojto, I reproduced the behavior you are talking about. I believe this is a different issue so I wrote it up as https://github.com/firebase/firebase-js-sdk/issues/6659.

Thanks for reporting

[avabhi]

Even I am also getting same issue with firebase version : 9.9.4

[dconeybe]

@vojto The "Document Already Exists" error you experienced is specific to transactions and was a bug that was accidentally introduced in v9.9.4. It will be fixed in the next release. See https://github.com/firebase/firebase-js-sdk/issues/6659 for details.

The "Document Already Exists" issue reported in the OP is not related to transactions, and still exists.

[appsgenie]

I am also seeing this with "firebase": "8.2.3" FirebaseError: [code=already-exists]: Document already exists Not using any transactions and simply doing firestore.collection('locations').add({..}) Seeing these sporadically in Sentry and no recent version updates have taken place. Wanted to see if it's ok to ignore these and if anything else might be happening.

[dconeybe]

@appsgenie That error is probably safe to ignore. What is likely happening is that the request is sent to the server to create the new document, then the client loses connection before the response is received from the server. When the client reconnects then it re-sends the "create document" request, which fails because while the document already exists because while the client was briefly offline the server completed the request to create the new document.