ray-llm
ray-llm copied to clipboard
ray-project
RayLLM - LLMs on Ray
# Purpose Aviary is an open source LLM management toolkit built on top of Ray Serve and Ray. LangChain is an incredibly popular open source toolkit for building LLM applications....
The following is the error encountered when trying to run `aviary run --model ./models/amazon--LightGPT.yaml` as per the readme setup of doing the following steps ```shell # Setup AWS env vars...
RWKV is shown in the main readme - https://github.com/ray-project/aviary#listing-all-available-models but is missing in the current models page - https://github.com/ray-project/aviary/tree/master/models
_This issue serves as a "news ticker" for the Aviary frontend._ **Current news:** We're sunsetting the non-Llama models as of the 0.3.0 release. The reason is because we've seen the...
Users aren't currently limited to one vote on 'best response' per prompt submitted or forced to 'change vote' to select a different option. Steps to reproduce: - Submit prompt -...
* Update models to pydantic v2 as latest vllm has adopted v2 models instead of v1 * Fix AutoscalingConfig model as it's from Ray Serve that is based on pydantic...
There is no release for 3 months and just few commits recently, so will this project be actively maintained? I tried serve using ray-llm with some LLM, and need to...
Has anyone had any success serving llms through the 0.5.0 docker image? I have followed the following steps: `cache_dir=${XDG_CACHE_HOME:-$HOME/.cache}` `docker run -it --gpus all --shm-size 1g -p 8000:8000 -e HF_HOME=/tmp/data...
