Bendr Radrigues

Support disabling of API response streaming via config

3

When using obscure LLMs/APIs that do not do streaming, it seems it is not possible for API just to return results in 1 go - there are 2 places in...

Issues loading model shards

4

folks, thank you for a very interesting project! i'm trying some basic scenarios and hit a few snags. Happy to split all to separate issues if needed. Setup: node/host1, ubuntu...