Langchain-Chatchat icon indicating copy to clipboard operation
Langchain-Chatchat copied to clipboard

Published 20 hours ago • verified publisher icon chatchat-space

如何配置 本地知识库

Open Yogier opened this issue 11 months ago • 6 comments

$ git lfs install $ git clone https://huggingface.co/THUDM/chatglm3-6b $ git clone https://huggingface.co/BAAI/bge-large-zh 知识库下载在本地后如何配置？

Yogier avatar Mar 17 '24 12:03 Yogier

你好 @Yogier！👋我是Dosu，一个友好的机器人，可以帮助你解决问题、回答问题以及成为一个贡献者。在等待人类维护者回复的同时，我在这里为你提供帮助。我已经注意到了你的问题，将会很快给你一个完整的回答。

dosubot[bot] avatar Mar 17 '24 12:03 dosubot[bot]

模型下载好后 在哪里添加且支持CPU环境下运行

Yogier avatar Mar 17 '24 12:03 Yogier

To support running the models downloaded from https://huggingface.co/THUDM/chatglm3-6b and https://huggingface.co/BAAI/bge-large-zh in a CPU environment in the Langchain-Chatchat repository, you should follow these steps:

  1. Download the models from the provided Hugging Face URLs.
  2. Set the MODEL_ROOT_PATH in the configs/model_config.py.example file to a directory where you want to store all your models. This can be an absolute path on your system.
  3. Place the downloaded models in the directory specified by MODEL_ROOT_PATH. Each model should be in its own directory within this root path. The directory names for the models should match the keys or values specified in the MODEL_PATH dictionary for them to be automatically detected and loaded by the program. For example:
    • For chatglm3-6b, create a directory named chatglm3-6b within MODEL_ROOT_PATH and place the downloaded model files there.
    • For bge-large-zh, create a directory named bge-large-zh within MODEL_ROOT_PATH and place the downloaded model files there.

Given the context from configs/model_config.py.example, the relevant sections to focus on are:

  • MODEL_ROOT_PATH: This should be set to the directory where you plan to store the models.
  • MODEL_PATH: This dictionary maps model names to their storage locations. If the model directory names match the keys in this dictionary and are placed under MODEL_ROOT_PATH, the program will automatically detect and load them.

Here is an example of how you might configure these settings:

MODEL_ROOT_PATH = "/path/to/your/model/directory"

MODEL_PATH = {
    "llm_model": {
        "chatglm3-6b": "path/to/chatglm3-6b",  # This path is relative to MODEL_ROOT_PATH or can be an absolute path
        "bge-large-zh": "path/to/bge-large-zh",  # This path is relative to MODEL_ROOT_PATH or can be an absolute path
        # other models...
    },
    # other model configurations...
}

Remember, the paths inside MODEL_PATH are examples. You should either place the models in directories that match the keys in MODEL_PATH under MODEL_ROOT_PATH or provide absolute paths to each model's directory.

For more details, you can refer to the configs/model_config.py.example file in the Langchain-Chatchat repository.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

dosubot[bot] avatar Mar 17 '24 12:03 dosubot[bot]

已经按照教程正确配置了本地模型，在CPU上运行，为何还会有下面的报错？

(langchain) [root@localhost Langchain-Chatchat]# python startup.py -a

==============================Langchain-Chatchat Configuration============================== 操作系统：Linux-3.10.0-1160.102.1.el7.x86_64-x86_64-with-glibc2.17. python版本：3.11.8 (main, Feb 26 2024, 21:39:34) [GCC 11.2.0] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b', 'zhipu-api', 'openai-api'] @ cpu {'device': 'cuda', 'host': '0.0.0.0', 'infer_turbo': False, 'model_path': 'chatglm3-6b', 'model_path_exists': True, 'port': 20002} {'api_key': '', 'device': 'auto', 'host': '0.0.0.0', 'infer_turbo': False, 'online_api': True, 'port': 21001, 'provider': 'ChatGLMWorker', 'version': 'glm-4', 'worker_class': <class 'server.model_workers.zhipu.ChatGLMWorker'>} {'api_base_url': 'https://api.openai.com/v1', 'api_key': '', 'device': 'auto', 'host': '0.0.0.0', 'infer_turbo': False, 'model_name': 'gpt-4', 'online_api': True, 'openai_proxy': '', 'port': 20002} 当前Embbedings模型： bge-large-zh-v1.5 @ cpu ==============================Langchain-Chatchat Configuration==============================

2024-03-17 21:13:36,369 - startup.py[line:655] - INFO: 正在启动服务： 2024-03-17 21:13:36,369 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 /root/Langchain-Chatchat/logs /root/anaconda3/envs/langchain/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃 warn_deprecated( 2024-03-17 21:13:44 | INFO | model_worker | Register to controller 2024-03-17 21:13:44 | ERROR | stderr | INFO: Started server process [3590] 2024-03-17 21:13:44 | ERROR | stderr | INFO: Waiting for application startup. 2024-03-17 21:13:44 | ERROR | stderr | INFO: Application startup complete. 2024-03-17 21:13:44 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:20000 (Press CTRL+C to quit) 2024-03-17 21:13:46 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker 9ac78b3e ... 2024-03-17 21:13:47 | WARNING | transformers_modules.THUDM.chatglm3-6b.9addbe01105ca1939dd60a0e5866a1812be9daea.tokenization_chatglm | Setting eos_token is not supported, use the default one. 2024-03-17 21:13:47 | WARNING | transformers_modules.THUDM.chatglm3-6b.9addbe01105ca1939dd60a0e5866a1812be9daea.tokenization_chatglm | Setting pad_token is not supported, use the default one. 2024-03-17 21:13:47 | WARNING | transformers_modules.THUDM.chatglm3-6b.9addbe01105ca1939dd60a0e5866a1812be9daea.tokenization_chatglm | Setting unk_token is not supported, use the default one. Loading checkpoint shards: 0%| | 0/7 [00:00<?, ?it/s] Loading checkpoint shards: 14%|██▌ | 1/7 [00:19<01:55, 19.26s/it] Loading checkpoint shards: 29%|█████▏ | 2/7 [00:41<01:45, 21.01s/it] Loading checkpoint shards: 43%|███████▋ | 3/7 [00:55<01:10, 17.74s/it] Loading checkpoint shards: 57%|██████████▎ | 4/7 [00:57<00:35, 11.67s/it] Loading checkpoint shards: 71%|████████████▊ | 5/7 [01:02<00:18, 9.15s/it] Loading checkpoint shards: 86%|███████████████▍ | 6/7 [01:15<00:10, 10.41s/it] Loading checkpoint shards: 100%|██████████████████| 7/7 [01:23<00:00, 9.75s/it] Loading checkpoint shards: 100%|██████████████████| 7/7 [01:23<00:00, 11.95s/it] 2024-03-17 21:15:11 | ERROR | stderr | 2024-03-17 21:15:11 | ERROR | stderr | Process model_worker - chatglm3-6b: 2024-03-17 21:15:11 | ERROR | stderr | Traceback (most recent call last): 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap 2024-03-17 21:15:11 | ERROR | stderr | self.run() 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/multiprocessing/process.py", line 108, in run 2024-03-17 21:15:11 | ERROR | stderr | self._target(*self._args, **self._kwargs) 2024-03-17 21:15:11 | ERROR | stderr | File "/root/Langchain-Chatchat/startup.py", line 389, in run_model_worker 2024-03-17 21:15:11 | ERROR | stderr | app = create_model_worker_app(log_level=log_level, **kwargs) 2024-03-17 21:15:11 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-03-17 21:15:11 | ERROR | stderr | File "/root/Langchain-Chatchat/startup.py", line 217, in create_model_worker_app 2024-03-17 21:15:11 | ERROR | stderr | worker = ModelWorker( 2024-03-17 21:15:11 | ERROR | stderr | ^^^^^^^^^^^^ 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/fastchat/serve/model_worker.py", line 77, in init 2024-03-17 21:15:11 | ERROR | stderr | self.model, self.tokenizer = load_model( 2024-03-17 21:15:11 | ERROR | stderr | ^^^^^^^^^^^ 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/fastchat/model/model_adapter.py", line 362, in load_model 2024-03-17 21:15:11 | ERROR | stderr | model.to(device) 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/transformers/modeling_utils.py", line 2595, in to 2024-03-17 21:15:11 | ERROR | stderr | return super().to(*args, **kwargs) 2024-03-17 21:15:11 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1160, in to 2024-03-17 21:15:11 | ERROR | stderr | return self._apply(convert) 2024-03-17 21:15:11 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^ 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 2024-03-17 21:15:11 | ERROR | stderr | module._apply(fn) 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 2024-03-17 21:15:11 | ERROR | stderr | module._apply(fn) 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply 2024-03-17 21:15:11 | ERROR | stderr | module._apply(fn) 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/torch/nn/modules/module.py", line 833, in _apply 2024-03-17 21:15:11 | ERROR | stderr | param_applied = fn(param) 2024-03-17 21:15:11 | ERROR | stderr | ^^^^^^^^^ 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1158, in convert 2024-03-17 21:15:11 | ERROR | stderr | return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) 2024-03-17 21:15:11 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-03-17 21:15:11 | ERROR | stderr | File "/root/anaconda3/envs/langchain/lib/python3.11/site-packages/torch/cuda/init.py", line 289, in _lazy_init 2024-03-17 21:15:11 | ERROR | stderr | raise AssertionError("Torch not compiled with CUDA enabled") 2024-03-17 21:15:11 | ERROR | stderr | AssertionError: Torch not compiled with CUDA enabled 2024-03-17 21:17:05 | INFO | stdout | INFO: 127.0.0.1:60504 - "GET / HTTP/1.1" 404 Not Found

Yogier avatar Mar 17 '24 13:03 Yogier

It looks like we need your specialized knowledge on this, @imClumsyPanda.

dosubot[bot] avatar Mar 17 '24 13:03 dosubot[bot]

当前启动的LLM模型：['chatglm3-6b', 'zhipu-api', 'openai-api'] @ cpu {'device': 'cuda', //此处cuda如何改为CPU

Yogier avatar Mar 17 '24 13:03 Yogier