Demistifying some configuration options

[g0di]

I'm building a JSON API using FastAPI which can takes up to 2.5GiB of RAM per request. My API is deployed on machines where resources are obviously limited. When there is too much requests at a time most requests start to fails because the process is going out of memory.

I would like to be able (if this is even possible) to limit the amount of parallel requests my API can handle at a time to avoid consuming too much memories.

However, I'm not really sure exactly how Hypercorn handle and dispatch the requests to the workers. I've had a look at configuration options and saw the following good candidates for that situation:

• h2_max_concurrent_streams
• max_app_queue_size
• backlog
• workers

I'm not an expert and options descriptions plus some tries on my side to change those did not helped me to achieve my goal.

Is there anyone able to provide some additional details on how those options affect Hypercorn so I can better setup my API to fit my needs?

Cheers!