Improving ApostropheCMS Performance with High Request Volumes and Large JSON Payloads

[maamounkhlifi]

We're currently experiencing performance challenges with our ApostropheCMS application, particularly when handling around 500-600 requests per minute (Rest API). The primary issue seems to stem from the transfer of large JSON payloads, as our models are complex, featuring numerous widgets and mixed relations.

Despite leveraging projection to trim down the data size, we're still facing slower response times than desired. This is especially true for data returned from widgets, such as the apostrophecms/image widget, where we ideally want to retrieve only the originalUrl instead of the entire object.

I'm looking for advice on the following:

1. Are there additional strategies within ApostropheCMS for enhancing performance under high request volumes?
2. Beyond projection, what other methods can effectively reduce the size of the data being transferred?
3. Specifically for widgets like apostrophecms/image, is there a way to limit the returned data to only essential information, such as originalUrl?

Any insights or suggestions on improving ApostropheCMS performance in such scenarios would be greatly appreciated.

[boutell]

Hi maamounkhlifi,

My first concern would be to prove it's definitely about the size of those JSON payloads. One way to do that is to use extendApiRoutes to manually trim them down aggressively before returning them, perhaps even to almost nothing just for the sake of testing under simulated load.

Also, what does your CPU load look like in this situation? Are you running multiple servers with a load balancer or is all of this traffic being served by a single process?

Also a good idea to check the CPU load on your mongodb server or cluster.

If you'd like to talk about having a full performance analysis of your application done by the Apostrophe development team, please do reach out to us and let the team know I sent you:

https://apostrophecms.com/contact-us

[BoDonkey]

Hi @maamounkhlifi - Did you manage to solve this issue? Is there any other assistance we can give, or can I close this?

[maamounkhlifi]

@boutell Thank you for the detailed response and suggestions! I have some updates on our situation that might change the analysis: Key Discovery: We discovered that we were stupidly running the application in development mode this entire time, which likely explains a significant portion of our performance issues. We're now in production mode.

Current Setup:

Running on Azure Premium v3 We had to manually trim the data in our responses to achieve better performance Traffic is being served by a single process (no load balancer currently)

Your Suggestions:

JSON Payload Trimming: Your suggestion about aggressively trimming payloads is exactly what we ended up doing manually. This did help significantly with performance. CPU Load: We haven't done a thorough analysis of CPU load during peak traffic yet, but this is definitely something we should monitor more closely now that we're in production mode. MongoDB Performance: MongoDB CPU load are good, we have a high performant cloud server.

Thanks again for the help!