[Feature Proposal] Start pools dynamically for a given origin

[PragTob]

(I promise this is the last issue for today! 💚 Thank you etc. etc.!)

Situation

Right now finch starts pools automatically when a new origin is encountered. The number of pools launched is static though (via count).

Problem

It can be tough to figure out what pool count (and size) to configure, as mentioned in #279. Esp. when being new to finch (and/or Req for that matter) and not really understanding these concepts.

Esp. if you don't know in advance where your HTTP requests will go and how many there will be (somewhat niche use case, but not totally i.e. proxies or web hooks), it's hard to configure which leads you to over provision the :default with big numbers for pool size and count that may be unnecessary for 99% of origins.

Solution

If we had an option that instead of raising when we couldn't get a connection we'd auto start a new pool to take care of that connection we could avoid these and allow pools to grow to their "natural"/needed size.

If #311 was fixed we could also allow pools to shrink again pool by pool (idle pools being shut down), i.e. the concern described there that if we shut down pools one by one we might end up with a too small pool count isn't a concern any more (as we'd then just start new pools again for the origin).

Options:

• auto_start?: to turn on auto starting additional pools for this origin/:default - i.e. turn on this proposed feature, might need to work on naming as auto starting the entire pool has a similarly named option
• initial_count (or just count): This would be how many pools we'd initially start for a new origin
• max_count: I guess a concern is that we'd start way too many pools, so providing a max_count option is probably a good idea

Implementation

In my head the implementation is somewhat simple? Just start a new one if we notice that the pool we selected is completely full. We could be more clever about this (#8) but I think it's good enough. Maybe the "realize pool is full/all connections are used" is more complicated though, I haven't looked yet.

As for selection strategy, right now we'd still end up over provisioning as pools are picked randomly so it's conceivable to end up with 10 pools each 5% full but since each one gets a new connection every now and then none would be shut down. That said, we can iterate on pool selection strategy or when to shut down a pool.

Also happy to try my hands at implementation, but again - I shouldn't be promising too much these days as my passion is usually bigger than my time 😂

[PragTob]

[sneako]

"Autoscaling" pools is definitely a feature I would be interested in. Since Finch started using lazy pools it almost behaves like this for my main use case, but obviously that doesn't work as well for the usage you describe (hitting uknown endpoints).

The code is already structured to easily allow for new pool selection strategies to be implemented, so that area is ready to be explored.

I agree that correctly implementing idle pool termination is probably important to finish before we are able to safely offer this feature.

Thanks for all of your recent input it has been very valuable and totally welcome, please don't hesitate to keep em coming!