router

router copied to clipboard

Describe the solution you'd like Wondering if it would be possible to lower the overall memory consumption of apollo router.

Describe alternatives you've considered Disabling features such as caching?

Additional context I am overall very happy with router and find it to be an improvement over apollo server. However, I was just a bit surprised seeing that it takes 250Mb of memory. Which is more then my apollo server instance used to take. Don't know if I am being nitpicky, just seems odd that it's consuming so much memory while using a very efficient language such as rust. Am I missing something, can I disable some feature which is responsible for such a high memory use?

For one other service written in Rust, we switched to jemalloc and could notice a strong improvement of memory consumption. We'll also try this with the router at some point.

For one other service written in Rust, we switched to jemalloc and could notice a strong improvement of memory consumption. We'll also try this with the router at some point.

I still don't think that this is the answer though, it is a potential solution to the problem, but it seems to be more of a band-aid then a cure. Yet again, I might be talking out of my ass as I have never really have dove deep into the overall router application and I have limited experience with rust. But I just feel that something isn't right. I'll try screwing around with the project in the next few days and I'll try pinpointing out exactly where the memory use is coming from.

we're considering the move to jemalloc too, I've seen it provide good results in other projects.

What is the context for the 250MB you measure? Virtual memory or RSS? At startup or after a few requests? Does it increase regularly, or does it stay at 250MB? Note that one instance of the router can handle the same load as multiple gateway instances, so a larger memory usage should be fine.

we're considering the move to jemalloc too, I've seen it provide good results in other projects.

What is the context for the 250MB you measure? Virtual memory or RSS? At startup or after a few requests? Does it increase regularly, or does it stay at 250MB? Note that one instance of the router can handle the same load as multiple gateway instances, so a larger memory usage should be fine.

It seems to be right on startup and RSS. Here is a screenshot from btop. I also used heap track and it seems that a large part of the memory is being held by a single function (unresolved function).

do you have a large schema?

@Geal I would say I do. Sadly can't share the schema with you for privacy reasons. However, to put it in context. The supergraph has 7401 lines. I still feel that the javascript version took less memory though. So I am still surprised that the memory consumption is this high

sure, that's mainly to provide a direction to investigate, thanks for the information

for those allocations in serde_v8, that's probably linked to the translation layer between the router and query planner. That's probably not something we will be able to fix anytime soon

@AdamJSoftware a large part of the memory consumption is due to the introspection data that is pregenerated and cached. If you do not care about introspection while in production, you can deactivate it in the configuration file like this:

server:
    introspection: false

@AdamJSoftware a large part of the memory consumption is due to the introspection data that is pregenerated and cached. If you do not care about introspection while in production, you can deactivate it in the configuration file like this:

server:
    introspection: false

I was about to mention that serde has less memory consumption then the unknown function but it was just ordered weird in that image. Regarding introspection, wow, it cut it in half. This is much more reasonable! Thanks for looking into it! I also think we should make this a bit more clear in the documentation as users might be doubling their memory for absolutely no reason.

Thanks for opening this originally. As of https://github.com/apollographql/router/pull/1517, we should be providing a more reasonable default with less initial usage and we'll get that cut into a release — likely this week.

In the general sense, we are certainly aware of other ways we can improve our memory footprint and motivated to keep it low, but for now we're going to close this to avoid it being a general purpose "lowering memory consumption" task. For example, we know that we will reduce the footprint further in the future as we divest from our V8 usage, but we need to build some underlying Rust tooling to completely get out of that business.

For the near-term, while we might find opportunities to improve, our priority won't be so much about getting our minimum footprint as low as possible but about building feature parity for Gateway users and keeping performance top of mind. Since performance wins can often be achieved through keeping objects in memory, we can't be certain that our memory usage won't increase — so this is not a guarantee of startup footprint — but we hope to deliver satisfactory improvements indefinitely in terms of how we utilize and what we ask of our host environment.

Thanks again, and watch out for the next release! (or try main today. 😉 )