Langchain-Chatchat icon indicating copy to clipboard operation
Langchain-Chatchat copied to clipboard

Published 20 hours ago • verified publisher icon chatchat-space

[BUG] 简洁阐述问题 / Concise description of the issue

Open RaspberrySun opened this issue 1 year ago • 4 comments

问题描述 / Problem Description 使用的模型是chatglm3-6b 执行python startup.py -a时报错: ==============================Langchain-Chatchat Configuration============================== 操作系统：Linux-6.2.0-39-generic-x86_64-with-glibc2.35. python版本：3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35 当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b'] @ cuda {'device': 'cuda', 'host': '0.0.0.0', 'infer_turbo': False, 'model_path': '/home/ubuntu/app/LangChain-model/chatglm3-6b', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： bge-large-zh-v1.5 @ cuda

附加信息 / Additional Information

2024-02-26 11:21:49 | ERROR | stderr | INFO: Started server process [242462] 2024-02-26 11:21:49 | ERROR | stderr | INFO: Waiting for application startup. 2024-02-26 11:21:49 | ERROR | stderr | INFO: Application startup complete. 2024-02-26 11:21:49 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:20000 (Press CTRL+C to quit) 2024-02-26 11:21:49 | INFO | model_worker | Register to controller There was a problem when trying to write in your cache folder (/root/.cache/huggingface/hub). You should set the environment variable TRANSFORMERS_CACHE to a writable directory. 2024-02-26 11:21:50 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker 2951c0a2 ... Could not locate the tokenization_chatglm.py inside THUDM/chatglm3-6b. 2024-02-26 11:21:50 | ERROR | stderr | Process model_worker - chatglm3-6b: 2024-02-26 11:21:50 | ERROR | stderr | Traceback (most recent call last): 2024-02-26 11:21:50 | ERROR | stderr | File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap 2024-02-26 11:21:50 | ERROR | stderr | self.run() 2024-02-26 11:21:50 | ERROR | stderr | File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run 2024-02-26 11:21:50 | ERROR | stderr | self._target(*self._args, **self._kwargs) 2024-02-26 11:21:50 | ERROR | stderr | File "/apps/svr/LLM/Langchain/Langchain-Chatchat/startup.py", line 389, in run_model_worker 2024-02-26 11:21:50 | ERROR | stderr | app = create_model_worker_app(log_level=log_level, **kwargs) 2024-02-26 11:21:50 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-02-26 11:21:50 | ERROR | stderr | File "/apps/svr/LLM/Langchain/Langchain-Chatchat/startup.py", line 217, in create_model_worker_app 如何加载本地模型，修改哪里的配置？

RaspberrySun avatar Feb 26 '24 03:02 RaspberrySun

🤖

嗨，@RaspberrySun！很高兴再次见到你。希望一切都好！

根据您提供的错误日志，问题似乎出在加载模型时无法找到tokenization_chatglm.py文件。这个文件是用于chatglm3-6b模型的分词器，它是加载和使用模型的必要部分。

要加载本地模型，您需要修改model_config.py文件中的MODEL_PATH和LLM_MODELS配置：

  1. MODEL_PATH：在这里指定本地模型的路径。如果模型目录的名称与MODEL_PATH中的键或值相同，程序将自动检测并加载它，您不需要修改MODEL_PATH中的路径。对于chatglm3-6b模型，您可以这样设置：
"llm_model": {
    "chatglm3-6b": "/path/to/your/local/model/chatglm3-6b",
    ...
}

将"/path/to/your/local/model/chatglm3-6b"替换为您本地chatglm3-6b模型的实际路径。

  1. LLM_MODELS：在这里指定项目启动时要加载的LLM模型。列表中的第一个模型将是API和WEBUI的默认模型。要使用chatglm3-6b模型，您可以这样设置：
LLM_MODELS = ["chatglm3-6b", ...]

确保"chatglm3-6b"在列表中。

如果在执行python startup.py -a时遇到错误，可能是由于GPU内存不足或模型路径不正确。请确保您的GPU有足够的内存来加载模型，且模型路径是正确的。

关于tokenization_chatglm.py文件的具体作用，我在Langchain-Chatchat代码库中没有找到确切的答案，但一般来说，这个文件应该包含了用于chatglm3-6b模型的特定分词器。这个分词器可能包含了一些特定的分词规则或者预处理步骤，这些都是模型正常工作所必需的。

如果您在本地模型的目录中没有找到tokenization_chatglm.py文件，可能需要从模型的原始源（如Hugging Face模型库）下载。如果模型的原始源也没有提供这个文件，那么可能需要联系模型的作者或者维护者获取。

参考资料：

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

dosubot[bot] avatar Feb 26 '24 03:02 dosubot[bot]

🤖

嗨，@RaspberrySun！很高兴再次见到你。希望一切都好！

根据您提供的错误日志，问题似乎出在加载模型时无法找到tokenization_chatglm.py文件。这个文件是用于chatglm3-6b模型的分词器，它是加载和使用模型的必要部分。

要加载本地模型，您需要修改model_config.py文件中的MODEL_PATH和LLM_MODELS配置：

  1. MODEL_PATH：在这里指定本地模型的路径。如果模型目录的名称与MODEL_PATH中的键或值相同，程序将自动检测并加载它，您不需要修改MODEL_PATH中的路径。对于chatglm3-6b模型，您可以这样设置：
"llm_model": {
    "chatglm3-6b": "/path/to/your/local/model/chatglm3-6b",
    ...
}

将"/path/to/your/local/model/chatglm3-6b"替换为您本地chatglm3-6b模型的实际路径。

  1. LLM_MODELS：在这里指定项目启动时要加载的LLM模型。列表中的第一个模型将是API和WEBUI的默认模型。要使用chatglm3-6b模型，您可以这样设置：
LLM_MODELS = ["chatglm3-6b", ...]

确保"chatglm3-6b"在列表中。

如果在执行python startup.py -a时遇到错误，可能是由于GPU内存不足或模型路径不正确。请确保您的GPU有足够的内存来加载模型，且模型路径是正确的。

关于tokenization_chatglm.py文件的具体作用，我在Langchain-Chatchat代码库中没有找到确切的答案，但一般来说，这个文件应该包含了用于chatglm3-6b模型的特定分词器。这个分词器可能包含了一些特定的分词规则或者预处理步骤，这些都是模型正常工作所必需的。

如果您在本地模型的目录中没有找到tokenization_chatglm.py文件，可能需要从模型的原始源（如Hugging Face模型库）下载。如果模型的原始源也没有提供这个文件，那么可能需要联系模型的作者或者维护者获取。

参考资料：

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

修改后仍然有问题

RaspberrySun avatar Feb 26 '24 04:02 RaspberrySun

🤖

Based on the error log you've provided, it seems like the tokenization_chatglm.py file is missing from your local chatglm3-6b model directory. This file is a part of the chatglm3-6b model and is required for the model to function correctly.

There are a few possible solutions to this issue:

  1. Ensure the chatglm3-6b model is correctly downloaded and placed in the correct directory. The tokenization_chatglm.py file should be included in the chatglm3-6b model when it's downloaded from the Hugging Face model hub. If it's not, you might need to download the model again. Make sure to place the downloaded model in the THUDM/chatglm3-6b directory under the MODEL_ROOT_PATH as specified in the model_config.py file. If MODEL_ROOT_PATH is not specified, you should specify an absolute path where all the models are stored. If the model directory name is the same as the key or value in MODEL_PATH, the program will automatically detect and load it, and you don't need to modify the path in MODEL_PATH. You can refer to the model_config.py.example file for more details.

  2. Check if there are any issues with how the file is being accessed. If the tokenization_chatglm.py file exists and is correctly placed, there might be an issue with how the file is being accessed. Make sure that the file is readable and that the directory is included in your system's PATH.

  3. Try using a different model. If the issue persists, you might want to consider using a different model that doesn't require a connection to 'huggingface.co', or download the model and use it locally.

  4. Download the model from a different source. As suggested in the chatglm3-6b模型加载失败 issue, you can try downloading the model from Modelscope or SwanHub using the following commands:

git lfs install
git clone https://www.modelscope.cn/ZhipuAI/chatglm3-6b.git

or

git lfs install
git clone https://swanhub.co/ZhipuAI/chatglm3-6b.git

Please try these solutions and let me know if any of them work for you. If you're still encountering issues, please provide more detailed information about your system setup and the exact error messages you're seeing.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

dosubot[bot] avatar Feb 26 '24 04:02 dosubot[bot]

问题描述 / Problem Description 使用的模型是chatglm3-6b 执行python startup.py -a时报错: ==============================Langchain-Chatchat Configuration============================== 操作系统：Linux-6.2.0-39-generic-x86_64-with-glibc2.35. python版本：3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35 当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b'] @ cuda {'device': 'cuda', 'host': '0.0.0.0', 'infer_turbo': False, 'model_path': '/home/ubuntu/app/LangChain-model/chatglm3-6b', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： bge-large-zh-v1.5 @ cuda

附加信息 / Additional Information

2024-02-26 11:21:49 | ERROR | stderr | INFO: Started server process [242462] 2024-02-26 11:21:49 | ERROR | stderr | INFO: Waiting for application startup. 2024-02-26 11:21:49 | ERROR | stderr | INFO: Application startup complete. 2024-02-26 11:21:49 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:20000 (Press CTRL+C to quit) 2024-02-26 11:21:49 | INFO | model_worker | Register to controller There was a problem when trying to write in your cache folder (/root/.cache/huggingface/hub). You should set the environment variable TRANSFORMERS_CACHE to a writable directory. 2024-02-26 11:21:50 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker 2951c0a2 ... Could not locate the tokenization_chatglm.py inside THUDM/chatglm3-6b. 2024-02-26 11:21:50 | ERROR | stderr | Process model_worker - chatglm3-6b: 2024-02-26 11:21:50 | ERROR | stderr | Traceback (most recent call last): 2024-02-26 11:21:50 | ERROR | stderr | File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap 2024-02-26 11:21:50 | ERROR | stderr | self.run() 2024-02-26 11:21:50 | ERROR | stderr | File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run 2024-02-26 11:21:50 | ERROR | stderr | self._target(*self._args, **self._kwargs) 2024-02-26 11:21:50 | ERROR | stderr | File "/apps/svr/LLM/Langchain/Langchain-Chatchat/startup.py", line 389, in run_model_worker 2024-02-26 11:21:50 | ERROR | stderr | app = create_model_worker_app(log_level=log_level, **kwargs) 2024-02-26 11:21:50 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-02-26 11:21:50 | ERROR | stderr | File "/apps/svr/LLM/Langchain/Langchain-Chatchat/startup.py", line 217, in create_model_worker_app 如何加载本地模型，修改哪里的配置？

应该是chatglm3-6b最新的tokenizer_config.json的问题：

  "auto_map": {
    "AutoTokenizer": [
      "THUDM/chatglm3-6b--tokenization_chatglm.ChatGLMTokenizer",
      null
    ]
  },

修改成

  "auto_map": {
    "AutoTokenizer": [
      "tokenization_chatglm.ChatGLMTokenizer",
      null
    ]
  },

Youngphone avatar Feb 26 '24 08:02 Youngphone

问题描述 / Problem Description 使用的模型是chatglm3-6b 执行python startup.py -a时报错: ==============================Langchain-Chatchat Configuration============================== 操作系统：Linux-6.2.0-39-generic-x86_64-with-glibc2.35. python版本：3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35 当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['chatglm3-6b'] @ cuda {'device': 'cuda', 'host': '0.0.0.0', 'infer_turbo': False, 'model_path': '/home/ubuntu/app/LangChain-model/chatglm3-6b', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： bge-large-zh-v1.5 @ cuda 附加信息 / Additional Information 2024-02-26 11:21:49 | ERROR | stderr | INFO: Started server process [242462] 2024-02-26 11:21:49 | ERROR | stderr | INFO: Waiting for application startup. 2024-02-26 11:21:49 | ERROR | stderr | INFO: Application startup complete. 2024-02-26 11:21:49 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:20000 (Press CTRL+C to quit) 2024-02-26 11:21:49 | INFO | model_worker | Register to controller There was a problem when trying to write in your cache folder (/root/.cache/huggingface/hub). You should set the environment variable TRANSFORMERS_CACHE to a writable directory. 2024-02-26 11:21:50 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker 2951c0a2 ... Could not locate the tokenization_chatglm.py inside THUDM/chatglm3-6b. 2024-02-26 11:21:50 | ERROR | stderr | Process model_worker - chatglm3-6b: 2024-02-26 11:21:50 | ERROR | stderr | Traceback (most recent call last): 2024-02-26 11:21:50 | ERROR | stderr | File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap 2024-02-26 11:21:50 | ERROR | stderr | self.run() 2024-02-26 11:21:50 | ERROR | stderr | File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run 2024-02-26 11:21:50 | ERROR | stderr | self._target(*self._args, **self._kwargs) 2024-02-26 11:21:50 | ERROR | stderr | File "/apps/svr/LLM/Langchain/Langchain-Chatchat/startup.py", line 389, in run_model_worker 2024-02-26 11:21:50 | ERROR | stderr | app = create_model_worker_app(log_level=log_level, **kwargs) 2024-02-26 11:21:50 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-02-26 11:21:50 | ERROR | stderr | File "/apps/svr/LLM/Langchain/Langchain-Chatchat/startup.py", line 217, in create_model_worker_app 如何加载本地模型，修改哪里的配置？

应该是chatglm3-6b最新的tokenizer_config.json的问题：

  "auto_map": {
    "AutoTokenizer": [
      "THUDM/chatglm3-6b--tokenization_chatglm.ChatGLMTokenizer",
      null
    ]
  },

修改成

  "auto_map": {
    "AutoTokenizer": [
      "tokenization_chatglm.ChatGLMTokenizer",
      null
    ]
  },

有用感谢！

VirgilG72 avatar Mar 09 '24 03:03 VirgilG72

Please try these solutions and let me know if any of them work for you. If you're still encountering issues, please provide more detailed information about your system setup and the exact error messages you're seeing.

有用感谢！！

imaodong avatar Mar 15 '24 05:03 imaodong