[Feature] Horizontal scaling of workloads

[Tan-tan-san]

Is your feature request related to a problem? Please describe. Its not necessarily a problem, but related to resource consumption and the need to horizontally scale the back end workloads. IE: API, metadata searches, webui, filesystem scans, ect.

Describe the solution you'd like Excluding the redis and DB deployments, the romm image has plenty going on under the hood that could make it hard to scale out for larger libraries. In my scenario I have Romm deployed in kubernetes and would like the ability to use HPA's to scale up and down 'worker pods' for the workloads mentioned above. I could keep the webui snappy and fast while heavy searches and updates were running in the back end.

Describe alternatives you've considered none

Additional context A good example for context is how immich-server deploys and can scale workloads out.

[gantoine]

@Tan-tan-san Interesting idea, but this is clearly out of the team's wheelhouse, so we'd need someone external to come in and take a crack at it, or at least lay out a plan/approach we can follow. Want to give it a shot?