FIRESTORE (10.12.0) INTERNAL ASSERTION FAILED: Unexpected state

[luke-rogers]

Operating System

iOS 17.4.1

Browser Version

Mobile Safari UI/WKWebView (AppleWebKit/605.1.15)

Firebase SDK Version

10.12.0

Firebase SDK Product:

Firestore

Describe your project's tooling

Angular web app running on iOS via Capacitor Firestore offline persistence enabled with unlimited cache size

Describe the problem

For a long time now we have been plagued by this iOS-only issue and have received many reports from users that it's causing the app to stop working and forcing them to restart.

We have captured some Firebase debug logs of this error which I am happy to share privately.

This seems similar to the following issues:

• #8068
• #7712
• #4451

Example Errors

• UnknownError: Attempt to get a record from database without an in-progress transaction
• UnknownError: Attempt to get all index records from database without an in-progress transaction
• UnknownError: Connection to Indexed Database server lost. Refresh the page to try again
• FIRESTORE (10.12.0) INTERNAL ASSERTION FAILED: Unexpected state

Example Stack Trace

Error: FIRESTORE (10.12.0) INTERNAL ASSERTION FAILED: Unexpected state
  at fail(node_modules/@firebase/firestore/dist/index.esm2017.js:197:36)
  at __PRIVATE_AsyncQueueImpl.Pu(node_modules/@firebase/firestore/dist/index.esm2017.js:18775:16)
  at __PRIVATE_AsyncQueueImpl.enqueue(node_modules/@firebase/firestore/dist/index.esm2017.js:18691:14)
  at __PRIVATE_AsyncQueueImpl.enqueueAndForget(node_modules/@firebase/firestore/dist/index.esm2017.js:18676:10)
  at n(node_modules/@firebase/firestore/dist/index.esm2017.js:14564:15)
  at <anonymous>(node_modules/@firebase/firestore/dist/index.esm2017.js:14540:7)
  at __PRIVATE_StreamBridge.yo(node_modules/@firebase/firestore/dist/index.esm2017.js:13897:10)
  at n(node_modules/@firebase/firestore/dist/index.esm2017.js:14145:81)
  at <anonymous>(node_modules/@firebase/firestore/dist/index.esm2017.js:14094:13)
  at ab(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:548:22)
  at F(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:516:9)
  at Z.prototype.ta(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:2162:5)
  at Rb(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:1035:129)
  at M.prototype.Y(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:904:50)
  at M.prototype.ca(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:832:35)
  at ab(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:548:22)
  at F(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:516:9)
  at Wc(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:1582:128)
  at h.bb(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:1579:5)
  at h.Ea(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:1576:61)
  at Lc(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:1474:50)
  at h.Pa(node_modules/@firebase/webchannel-wrapper/dist/webchannel-blob/esm/webchannel_blob_es2018.js:1440:27)
  at {_ZoneImpl#2}.run(node_modules/zone.js/fesm2015/zone.js:111:37)
  at <anonymous>(node_modules/zone.js/fesm2015/zone.js:2299:30)
  at {_ZoneImpl#2}.runTask(node_modules/zone.js/fesm2015/zone.js:157:39)
  at drainMicroTaskQueue(node_modules/zone.js/fesm2015/zone.js:529:23)

Steps and code to reproduce issue

Steps currently unknown but possibly related to the app going between the background and the foreground over time.

[luke-rogers]

Looking into the Firebase logs, the error seems to occur after updateClientMetadataAndTryBecomePrimary is called.

Our app uses persistentSingleTabManager(undefined) for the tabManager config in the Capacitor environments as there can never be more than 1 tab. I assume this tab manager is the cause of the many PUT clientMetadata and SimpleDb GET owner owner logs.

I wonder if there would be any legs in not having a tab manager if that will result in no calls to updateClientMetadataAndTryBecomePrimary etc.

This is also similar to what is described by another GitHub user here

I don't know a lot about the internals of the tab management so might be completely wrong here.

Sample Log

[2024-05-16T13:03:08.238Z] Firestore (10.12.0): SimpleDb PUT clientMetadata <auto-key> {"clientId":"jtQjaM2I0v6CbepACBf7","updateTimeMs":1715864588238,"networkEnabled":true,"inForeground":false}
[2024-05-16T13:03:08.242Z] Firestore (10.12.0): SimpleDb GET owner owner {"ownerId":"jtQjaM2I0v6CbepACBf7","allowTabSynchronization":false,"leaseTimestampMs":1715864584009}
[2024-05-16T13:03:08.244Z] Firestore (10.12.0): SimpleDb GET owner owner {"ownerId":"jtQjaM2I0v6CbepACBf7","allowTabSynchronization":false,"leaseTimestampMs":1715864584009}
[2024-05-16T13:03:08.244Z] Firestore (10.12.0): IndexedDbPersistence Client 'jtQjaM2I0v6CbepACBf7' is not zombied in LocalStorage
[2024-05-16T13:03:08.244Z] Firestore (10.12.0): SimpleDb PUT owner owner {"ownerId":"jtQjaM2I0v6CbepACBf7","allowTabSynchronization":false,"leaseTimestampMs":1715864588244}
[2024-05-16T13:03:25.878Z] Firestore (10.12.0): IndexedDbPersistence Starting transaction: updateClientMetadataAndTryBecomePrimary
[2024-05-16T13:03:48.787Z] Firestore (10.12.0): SimpleDb Aborting transaction: Attempt to get a record from database without an in-progress transaction
[2024-05-16T13:03:48.787Z] Firestore (10.12.0): SimpleDb Transaction failed with error: Attempt to get a record from database without an in-progress transaction Retrying: true
[2024-05-16T13:03:48.787Z] Firestore (10.12.0): SimpleDb Opening database: firestore/[DEFAULT]/insite-teams/main
[2024-05-16T13:03:48.802Z] Firestore (10.12.0): WebChannelConnection RPC 'Listen' stream 0x7d7f4c65 received: {"targetChange":{"resumeToken":"CgkIic7l5p2ShgM=","readTime":"2024-05-16T13:03:34.823689Z"}}
[2024-05-16T13:03:56.239Z] Firestore (10.12.0): SimpleDb Aborting transaction: Attempt to get a record from database without an in-progress transaction
[2024-05-16T13:03:56.239Z] Firestore (10.12.0): SimpleDb Transaction failed with error: Attempt to get a record from database without an in-progress transaction Retrying: true
[2024-05-16T13:03:56.239Z] Firestore (10.12.0): SimpleDb Opening database: firestore/[DEFAULT]/insite-teams/main
[2024-05-16T13:03:56.243Z] Firestore (10.12.0): SimpleDb GET owner owner {"ownerId":"jtQjaM2I0v6CbepACBf7","allowTabSynchronization":false,"leaseTimestampMs":1715864588244}
[2024-05-16T13:03:56.243Z] Firestore (10.12.0): SimpleDb PUT clientMetadata <auto-key> {"clientId":"jtQjaM2I0v6CbepACBf7","updateTimeMs":1715864636243,"networkEnabled":true,"inForeground":false}
[2024-05-16T13:03:56.244Z] Firestore (10.12.0): SimpleDb GET owner owner {"ownerId":"jtQjaM2I0v6CbepACBf7","allowTabSynchronization":false,"leaseTimestampMs":1715864588244}
[2024-05-16T13:03:56.245Z] Firestore (10.12.0): SimpleDb GET owner owner {"ownerId":"jtQjaM2I0v6CbepACBf7","allowTabSynchronization":false,"leaseTimestampMs":1715864588244}
[2024-05-16T13:04:02.786Z] Firestore (10.12.0): SimpleDb Aborting transaction: Attempt to get records from database without an in-progress transaction
[2024-05-16T13:04:02.786Z] Firestore (10.12.0): SimpleDb Transaction failed with error: Attempt to get records from database without an in-progress transaction Retrying: false
[2024-05-16T13:04:02.788Z] Firestore (10.12.0): INTERNAL UNHANDLED ERROR:  Attempt to get records from database without an in-progress transaction
[2024-05-16T13:04:19.979Z] Firestore (10.12.0): WebChannelConnection RPC 'Listen' stream 0x7d7f4c65 received: {"targetChange":{"resumeToken":"CgkIrpqg/J2ShgM=","readTime":"2024-05-16T13:04:19.823918Z"}}
[2024-05-16T13:04:19.989Z] Firestore (10.12.0): FIRESTORE (10.12.0) INTERNAL ASSERTION FAILED: Unexpected state

@wu-hui I would be Happy to share the full log with you privately if you could let me know the best way to do that.

[vojto]

This has gotten really bad for us over the past few days. We didn't deploy anything.

Edit: This is on desktop.

[luke-rogers]

@vojto Perhaps that's a different issue, this has always been limited to just iOS for us.

[wu-hui]

I think the multitab log might be unrelated, this seems to stem from a underlying webchannal message.

[luke-rogers]

Fair enough, not an area I've dug too deep into.

@wu-hui what steps can I take to help try and move this forward?

[luke-rogers]

Still seeing this issue on the latest version. I have logs I can share if that would be useful?

[MarkDuckworth]

@luke-rogers I would like to review the logs. You can share the logs through a private GitHub repo (shared with me), through a Cloud support ticket, or through a Firebase support ticket. If creating a ticket with either Google support channel, please reference this GitHub issue so the support staff knows to route the ticket to my team. Thanks!

[SamLoys]

I encounter the same problem. For me this happens in my webapp running in a webview through Flutter.

Usually this is combined with QuotaExceededError __PRIVATE_WebStorageSharedClientState.setItem(@firebase/firestore/dist/index.esm2017)

Failed to execute 'setItem' on 'Storage': Setting the value of 'firestore_clients_firestore/[DEFAULT]/kekkle/_jLkv2Vnt89NL5Xx3UYtN' exceeded the quota.

The problem is always resolved by loggin out (clearing local storage). So my first guess is that the quota exception is not handled correcrly

EDIT: I created a new issue, as I am not sure if the underlying problem is the same.

[luke-rogers]

@MarkDuckworth I've shared the logs through a Firebase support ticket and via a private repo

Let me know if you need anything more from me.

[luke-rogers]

@MarkDuckworth did you get access to the logs? Support said they forwarded them on.

[MarkDuckworth]

@luke-rogers, I received the logs. Thank you for sharing. On initial review, I don't see an obvious cause, but we will give it some deeper investigation as soon as we can.

I've also started work on a PR to help us get error codes from these internal assertion failed messages.

Do you also have a reproducible example that you can share with us?

[luke-rogers]

@MarkDuckworth Thanks for the update.

PR looks good. Hopefully, you will be able to merge that. We would be happy to test a pre-release version of this if that would still be helpful?

Unfortunately, we do not have a way of reproducing this ourselves at the minute.

I have added some links to stack traces from Sentry to the readme of private repo I shared with you.

[tom-andersen]

@luke-rogers Mark is out of office and asked I take a look at logs. Could you grant access to private repo, to myself @tom-andersen and maybe some other members of our team such as @wu-hui ? Thanks.

[luke-rogers]

@tom-andersen added yourself and @wu-hui to the repo.

Still struggling with a reproduction but if there is anything else I can try and help with please let me know.

[tom-andersen]

Update:

The error with text database without an in-progress transaction is from within the IndexedDB implementation. This becomes a fatal error when Firestore settings disable multi-tab support (as far as I can tell from reading code).

The workaround might be to enable multi-tab support, since Firestore SDK will then handle this error by entering a non-primary role until IndexedDB resolves the problem, instead of treating this like a fatal error.

That this happens at all, is a little concerning, and might have more to do with the AppleWebKit.

[tom-andersen]

@luke-rogers Are you able to enable multi-tab and see if that resolves the problem?

[luke-rogers]

@tom-andersen we will enable multi-tab and report back. Hopefully doing a release today so should have some initial feedback by the end of the week.

[google-oss-bot]

Hey @luke-rogers. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[luke-rogers]

We released yesterday with the change to enable multi-tab. We will have more of an update within the next few days as to whether it's made a difference.

[luke-rogers]

Unfortunately enabling multi-tab has not resolved this issue. I am still trying to access the impact on the end user experience but we have managed to capture some logs for the error. I've added those to the GitHub repo previously shared.

The new log files for this error with Firebase version 10.12.4 and multi-tab enabled are:

1. 2024-8-1_17-6-47.log
2. 2024-8-1_8-42-34.log

[luke-rogers]

Added an additional log file for Firebase version 10.12.5: 2024-8-12_12-53-14.log

[braincomb]

Just wanted to chime in and mention that we also started seeing the same error in our Capacitor app, running on Android.

One notable change we did that started showing this error is when we introduced enableNetwork() in an attempt to fix hanging onSnapshot listeners when app is backgrounded. I think we are still seeing that issue, but we sporadically started seeing this error too, although it's very hard to replicate. Still uncertain of the impact it has on the user experience.

Some details:

• Firebase JS SDK v10.13.0
• Not running indexedDbPersistence, just the memoryLocalCache which is default I believe

[anderecc]

Hi, sorry about the English, it's not my mother tongue

I was getting the error “@firebase/firestore: Firestore (10.13.0): FIRESTORE (10.13.0) INTERNAL ASSERTION FAILED: Unexpected state”, when I went to investigate I saw that the cause would be the error ‘ERR_BUFFER_OUT_OF_BOUNDS’, what helped me to solve this error was this article in Stackoverflow, I downgraded nodejs from version 22.7.0 to 22.5.0 and it solved my problem, I hope it helps someone.

[ElianCordoba]

I my particular case the solution was to downgrade to the version 10.6.0, 10.7.0 is where the issue start. Here is the diff. If I got time I'll try to debug the SDK locally to see if I can figure out what the error is

[braincomb]

I want to add on an interesting observation, since I've seen a few issues pop up relating to Capacitor which means it's a mobile app. My friend was able to coincidentally reproduce it where he had a very bad internet connection. And he saw our handled Firestore errors, but Sentry reported them as ASSERTION FAILED.

Interesting ....

[braincomb]

@luke-rogers @vojto curious if you've been able to fix this or find out the cause of it? These errors are plaguing our Sentry from our Capacitor app.

[shaffy9633]

@anderecc would it work with nodejs v20.11.0? I see some of my users getting FIRESTORE (10.12.4) INTERNAL ASSERTION FAILED: Unexpected state

[anderecc]

@shaffy9633, Hello, I believe it will work, I'm using nodejs 22.9.0 and firestore 10.13.0 and not having any problems at the moment related to this 'error', try updating firestore

[TimmNL]

I have a similar issue (https://github.com/firebase/firebase-js-sdk/issues/8383) and tried to implement the new version of the sdk released yesterday, 11.0.0. Sadly, the problem is not fixed, as I still get @firebase/firestore: Firestore (11.0.0): FIRESTORE (11.0.0) INTERNAL ASSERTION FAILED: Unexpected state.