Automattic
Document Init Performance Bottleneck
Prerequisites
- [X] I have written a descriptive issue title
- [X] I have searched existing issues to ensure the performance issue has not already been reported
Last performant version
(last two/three years)
Slowed down in version
5.10.18
Node.js version
16.14.0
🦥 Performance issue
We query a large number of documents without using .lean() in some of our express.js GET endpoints. Those documents contain some subdocument properties.
Our HTTP GET Endpoints spend about 83% of their time with synchronous operations on the CPU which block the event loop.
In particular the Document.init method has a large total time (see APM screenshot below).
The clone method has a large self time (see Tracing screenshot below).
APM:
Local Node Tracing:
Steps to Reproduce
I am currently working on a small repo to reproduce this issue. Just wanted to post the timing analysis in advance in case someone has seen the same behaviour in APM tools.
Expected Behavior
Document.init should not take that long. The performance is currently not acceptable. We have to use .lean() on all GET endpoints otherwise our endpoints are slowed down x30 (from 100ms to 3000ms on average).
Have you confirmed that this slowness started in exactly 5.10.18? Meaning that 5.10.17 is much faster for you?
Oh, no that is not what I mean by the report above. We already experienced these slowdowns in 5.10.9 and before. I guess it is because of our schema size and number of objects. Thus my question: Are these typical timings when using mongoose without lean on GET endpoints which return a lot of data? If so, I think it is not possible to use mongoose on GET endpoints without .lean() because the performance of successive Document.init is so bad.
If you can come up with a standalone repro script that demonstrates a case where Document.init() performance is extremely slow, that would help us fix this issue. We've been working hard on improving Mongoose's performance over the last year, and we've made some progress. Having more scripts to benchmark against would help us make more progress.
We will look into this over the next couple of weeks at Nesto. I will post the repro!
We are also seeing this issue, around 30% on our case. Tested against v5 and v6,
[email protected] [email protected]
@MartinLoeper did you manage to solve the issue or your side?
Hey @fredericosilva, thanks for sharing your numbers. We recently compared v5 to v6 and found a huge performance improvement (like x7 for non-lean documents). We are still investigating the issue. However, it lost a bit of priority because we know that updating to v6 has so much potential.