onSnapshot performance issue when fetching >1.5k documents and under load stress

[choxnox]

[REQUIRED] Describe your environment

• Operating System version: MacOS 12.4
• Browser version: Chrome v104
• Firebase SDK version: 8.10.1
• Firebase Product: Firestore

[REQUIRED] Describe the problem

We have a chat system where users can send messages as well as emoji reactions. We've structured the app in a way that we use the same collection for both and we differentiate between messages by their type (default and reaction) and we separated app logic accordingly (two separate onSnapshot subscription hooks for messages and reactions).

We're investigating the performance bandwidth by simulating 1000 users sending 1 reaction in a time span of 10 seconds (essentially ~100 reactions/second) and what we've noticed is that there is a significant drop in performance in such scenario.

The performance drop is pretty obvious in UI for both laptops and mobiles (with mobiles being even more obvious due to their hardware configurations): animations become laggy and the whole UI behaves as if CPU is under a lot of stress (typing something in the input shows definite lag in the response) until the client side received all snapshot updates

We've dug into our code thinking we might've missed something within our React code but going deeper and deeper we've realized we can replicate exactly the same scenario in a very basic isolated code.

We also went through all related issues in this repository and there are two which seem to be very similar but they have been dealt with long time ago: https://github.com/firebase/firebase-js-sdk/issues/2668 and https://github.com/firebase/firebase-js-sdk/issues/2620.

For the purpose of demonstrating this issue we've created a very basic demo here: https://k0j9iq.csb.app

For visualization purposes we've added a dummy animation - in our app we use animated emoji instead but the same performance drop issue can be observed in the demo link: when the page is first loaded, for a split of a second the animation goes well, then it slows down until we receive the initial 1.7k chat messages and once the button which added 1000 emoji reaction documents to the collection is clicked, the animation slows down for the next 10-20 seconds until the client side receives all document updates after which the animation and the whole UI continues to run at normal speed.

Doing performance profiling in the Chrome shows that the CPU time is being spent inside Firebase SDK:

[MarkDuckworth]

Thanks for bringing this to our attention. We will look into it and follow up with any questions we have.

[choxnox]

I hope this gets higher priority attention soon.

While I understand you might be busy with on-going tasks, it would be great if you could prioritize this issue since fetching 1.5k documents at once (and only one time) shouldn't cause issues which originate from the SDK itself as it defeats the purpose of Firestore being real time DB when an issue like this seem to hinder app's scalability, if you know what I mean.

[MarkDuckworth]

Wanted to follow up on this. I've looked at your repro and while I don't see the slow down, either on a phone or a laptop, I can understand why you feel this is significant.

With respect to 100 mutations per second, that may be pushing the use case for this SDK. If you do need to achieve that throughput, can you move this code into a web worker? Alternatively, frequent state syncing is listed as a fit for RTDB and might be worth looking at.

The performance drop you report on the initial load of 1.7k documents is more concerning to me, but I'm not seeing significant performance degradation like in the old issues you linked. Alternative to loading all messages at once, would your use case allow you to load the documents in a paged approach, for example using limit(...) and offset(...)

[google-oss-bot]

Hey @choxnox. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[google-oss-bot]

Since there haven't been any recent updates here, I am going to close this issue.

@choxnox if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.