infiniflow
[Question]: what base URL should be used when using Xinference to deploy local models
Describe your problem
I use Xinference (or ollama) to deploy local llm models I can download glm4-chat-1m from Xinference, or local llm custom-glm4-chat and I can enter the UI of the conversation, and the conversation is successful, but I can't add it to ragflow, which basic URL is right. or I did wrong things. Here's the URL I've tried, and the error info
"http://host.docker.internal:9997/v1" --- "提示 : 102--Fail to access model(glm4-chat-1m).ERROR: Connection error." "http://127.0.0.1:9997/v1" --- same sith above "http://host.docker.internal:9997" --- same with above "http://127.0.0.1:9997" --- same with above "http://localhost:9997" --- same with above "http://localhost:9997/v1" --- same with above
“http://host.docker.internal:11434”--- 提示 : 102 Fail to access model(llama3).ERROR: [Errno -2] Name or service not known
Hi, I have encountered a similar issue. Could you please let me know if you have resolved this problem? Thanks a lot!
Hi, I have encountered a similar issue. Could you please let me know if you have resolved this problem? Thanks a lot!
not yet，you can browse other issues. I guess ragflow just doesn't fit linux system. windows maybe ok
Hi, I have encountered a similar issue. Could you please let me know if you have resolved this problem? Thanks a lot!
not yet，you can browse other issues. I guess ragflow just doesn't fit linux system. windows maybe ok
Thank you for your prompt reply! Actually, I'm using windows. But it's still not working :(
derppp. this is dead in the water if we cant connect to local LLMs, i have had no luck with Olama or LM Studio tried everything under the sun.
None from me, I looked hard atthe code. Doesn't seems to exist in the back end for connecting to LM Studio. And no luck with olama. It's a shame. Had my hopes up for this tool. Maybe one day.
I also use xinference, in your case it should be http://localhost:9997/v1
