zerotoone01

> qwen1.5移除了chat和chat_stream，可以参考https://qwen.readthedocs.io/en/latest/inference/chat.html 只要修改bridge_qwen_local.py中的llm_stream_generator即可 > > ``` > device = get_conf('LOCAL_MODEL_DEVICE') > system_prompt = get_conf('INIT_SYS_PROMPT') > ``` > > ``` > def llm_stream_generator(self, **kwargs): > def adaptor(kwargs): > query = kwargs['query']...

before ollama start, run: export OLLAMA_HOST="0.0.0.0:11434" v37.4版本是可以直接支持的，改config的几个点即可： config.py API_KEY = "ollama-key" LLM_MODEL = "one-api-qwen:14b(max_token=32768)" API_URL_REDIRECT = {"https://api.openai.com/v1/chat/completions": "http://your_ip:11434/v1/chat/completions"} AVAIL_LLM_MODELS = ["one-api-qwen:14b(max_token=32768)"] CUSTOM_API_KEY_PATTERN = "ollama-key"