flame icon indicating copy to clipboard operation
flame copied to clipboard

Allow callers to define the scaling strategy of the FLAME pool

Open nickdichev-firework opened this issue 6 months ago • 2 comments

I've been evaluating FLAME to replace the lambda architecture we are using for one of our services. We are currently using Kubernetes jobs as the runner. We have explicitly chosen to make the job/lambda call one-to-one due to the constraints of our application's requirements.

As such, in my first few experiments with FLAME I set the :max_concurrency option to 1 and :single_use to true. This works okay, however, due to the logic of the existing "max concurrency per runner" strategy, it causes us to get penalized waiting for a new k8s pod to spin up more often than we can tolerate.

The strategy we are trying to implement is "constant number of overprovisioned workers". I think the behavior I've extracted is flexible enough for all callers to be able to define their own strategies, however, I am definitely looking for feedback there.

There are still some rough edges in the PR, but I was hoping to see what you all think about this general direction.

For reference, here's the strategy module I've implemented in my app: https://gist.github.com/nickdichev-firework/e24530a6a36833c9c5aff8fb9ed8f970

nickdichev-firework avatar Aug 18 '24 20:08 nickdichev-firework