web-ui icon indicating copy to clipboard operation
web-ui copied to clipboard

how to use the LLM that deploy in LAN network by http:x.x.x.x:port?

Open Firegolden opened this issue 9 months ago • 1 comments

Hi,

I have a deepseekv3 model API base on interor network, like http:x.x.x.x:port. i want make it work for browser use web ui.

make the code follow changed in there:

"deepseek": ["deepseek-chat", "deepseek-reasoner","deepseek-v3"],
        if kwargs.get("model_name", "deepseek-chat") == "deepseek-reasoner":
            return DeepSeekR1ChatOpenAI(
                model=kwargs.get("model_name", "deepseek-reasoner"),
                temperature=kwargs.get("temperature", 0.0),
                base_url=base_url,
                api_key=api_key,
            )
        elif kwargs.get("model_name", "deepseek-chat") == "deepseek-v3":
            return DeepSeekR1ChatOpenAI(
                model=kwargs.get("model_name", "deepseek-v3"),
                temperature=kwargs.get("temperature", 0.0),
                base_url=base_url,
                api_key=api_key,
            )

But the error happened when runing: ···

While trying to retrieve the URL:
http://x.x.x.x:1234/v1/chat/completions
    <div class="line">
      <div class="label">The content could not be delivered due to the following condition:</div>
      <div class="desc">
        Target service not allowed
      </div>
    </div>

···

what's the problem?

Firegolden avatar Mar 28 '25 05:03 Firegolden

@Firegolden change endpoint to :http://x.x.x.x:1234/v1, it should be ok

Annisun2016 avatar Apr 16 '25 03:04 Annisun2016