Fedir Skitsko
Fedir Skitsko
+1
Just a sample like this ```python from langchain.llms import Aviary llm = Aviary(model='TheBloke/Llama-2-70B-chat-AWQ', aviary_url="http://localhost:8000/v1", aviary_token="EMPTY") output = llm('How do you make fried rice?') ``` Gives me an error ` http://localhost:8000/v1...
Does anybody know the solution? Thanks
I've upgraded to latest ray/ray-llm and now having this issue. Is there a known hot fix? `gpu_memory_utilization` doesn't help at all. Trying to run TheBloke/70b with this config ```yaml engine_config:...
facing the same issue with ```bash rye add "ray[default]" error: did not find package 'ray[default]' without using pre-releases. ``` any ideas when it will be fixed in the release?
were you able to run this locally? does it work? I am just looking forward to see how to update this project to support latest vllm
I previously raised a question in Slack community channel regarding ongoing support for this project. About a month ago, there was a discussion promising continued development and updates. However, I...
@xwu99 is heavily working on updates, let's 🤞 and see the progress here #149
same issue here, losing DNS on 1.30 even without `exit-node` usage, running tailscale in the docker (github actions https://github.com/tailscale/github-action) ``` tailscale netcheck Report: * UDP: false * IPv4: (no addr...