flame
flame copied to clipboard
Allow callers to define the scaling strategy of the FLAME pool
I've been evaluating FLAME to replace the lambda architecture we are using for one of our services. We are currently using Kubernetes jobs as the runner. We have explicitly chosen to make the job/lambda call one-to-one due to the constraints of our application's requirements.
As such, in my first few experiments with FLAME I set the :max_concurrency
option to 1 and :single_use
to true. This works okay, however, due to the logic of the existing "max concurrency per runner" strategy, it causes us to get penalized waiting for a new k8s pod to spin up more often than we can tolerate.
The strategy we are trying to implement is "constant number of overprovisioned workers". I think the behavior I've extracted is flexible enough for all callers to be able to define their own strategies, however, I am definitely looking for feedback there.
There are still some rough edges in the PR, but I was hoping to see what you all think about this general direction.
For reference, here's the strategy module I've implemented in my app: https://gist.github.com/nickdichev-firework/e24530a6a36833c9c5aff8fb9ed8f970