ray-llm icon indicating copy to clipboard operation
ray-llm copied to clipboard

Remote address refuse queries

Open rifkybujana opened this issue 1 year ago • 1 comments

Hi, I've managed to deploy my models using AWS as instructed in the README.md. It works well by using the http://localhost:8000/v1 endpoint inside the cluster. However, when I use http://<remote head server public ip>:8000/v1, it always refuses connection to any query. There are no problems accessing the dashboard or Prometheus using the same public IP. Do you know what I am doing wrong?

rifkybujana avatar Nov 30 '23 11:11 rifkybujana

Can you provide the code you are using for querying?

akshay-anyscale avatar Dec 14 '23 19:12 akshay-anyscale