fermyon
Component eviction / lazy loading
This suggestion came from a discussion with a Spin user who had expected Spin to have a smaller memory footprint when the process is not servicing requests. They expected that the process would start listening for requests upon startup while not loading any components, as requests are received instances of the handling component would be created to handle them, then after a period of time the app would evict the components returning back to the svelte listening state.
This seems like it would be trading space for time - reduce memory footprint but need to reload/recompile idle components. Did the user share any further info on the memory impact? I would expect the operating system to swap out memory that isn't getting touched if it's needed for other purposes...
I can provide some additional context. The user also was running a .NET workload using native Linux containers in Kubernetes and made the comparison between their Wasm application and the .NET native application when they were both listening w/o any requests incoming. They observed that the memory footprint was nearly the same. Their expectation was that the Wasm workload would be significantly smaller that the .NET process.
The user was also interested in scale to zero scenarios in Kubernetes; having no instances running when there were no incoming requests. We talked a bit about how, at some point, there needs to be a process listening either outside to the pod workload or pod workload itself to be able to handle an incoming request. As a next best solution, they suggested that the Spin process should lazily evict workloads that were not being used to optimize for resources on a given node and combine that strategy with horizontal pod autoscaling and cluster autoscaler / karpenter.
Thanks for the extra info! It seems like the first step would be to benchmark how much memory the components are occupying compared to how much memory an inactive Spin process-set occupies (I say "set" because a running Spin instance comprises a launcher process and 1 or more trigger processes); that would allow us to evaluate how helpful eviction would be in such a scenario.
I'd still like to get some understanding of what the observation involved. Was the user experiencing memory pressure causing swapping or OOM-killing? Was the idle process occupying physical memory and not being swapped out by the operating system despite memory pressure? I'm not familiar with how Linux/k8s handles these things.
I found an older issue asking for a similar feature:
https://github.com/fermyon/feedback/issues/50
