ray-llm
ray-llm copied to clipboard
Remote address refuse queries
Hi, I've managed to deploy my models using AWS as instructed in the README.md. It works well by using the http://localhost:8000/v1
endpoint inside the cluster. However, when I use http://<remote head server public ip>:8000/v1
, it always refuses connection to any query. There are no problems accessing the dashboard or Prometheus using the same public IP. Do you know what I am doing wrong?
Can you provide the code you are using for querying?