bifrost-gateway
bifrost-gateway copied to clipboard
limiter to prevent oom
temporary attempt at identifying where the OOM is.
There are not so many requests presently to make it clear why staging is repeatedly running out of memory.
This limits how many active requests/ caches are allowed, to see if the issue is that there are enough 'large' requests that end up stacking concurrently to cause the issue.