[BUG] 简洁阐述问题 / 运行python init_database.py --recreate-vs错误

[YanzyUpEveryDay]

问题描述 / Problem Description 用简洁明了的语言描述这个问题 / Describe the problem in a clear and concise manner.

复现问题的步骤 / Steps to Reproduce

1. 执行 '...' / Run '...'
2. 点击 '...' / Click '...'
3. 滚动到 '...' / Scroll to '...'
4. 问题出现 / Problem occurs

预期的结果 / Expected Result 描述应该出现的结果 / Describe the expected result.

实际结果 / Actual Result 描述实际发生的结果 / Describe the actual result.

环境信息 / Environment Information

• langchain-ChatGLM 版本/commit 号：(例如：v2.0.1 或 commit 123456) / langchain-ChatGLM version/commit number: (e.g., v2.0.1 or commit 123456)
• 是否使用 Docker 部署（是/否）：是 / Is Docker deployment used (yes/no): yes
• 使用的模型（ChatGLM2-6B / Qwen-7B 等）：ChatGLM-6B / Model used (ChatGLM2-6B / Qwen-7B, etc.): ChatGLM2-6B
• 使用的 Embedding 模型（moka-ai/m3e-base 等）：moka-ai/m3e-base / Embedding model used (moka-ai/m3e-base, etc.): moka-ai/m3e-base
• 使用的向量库类型 (faiss / milvus / pg_vector 等)： faiss / Vector library used (faiss, milvus, pg_vector, etc.): faiss
• 操作系统及版本 / Operating system and version:
• Python 版本 / Python version:
• 其他相关环境信息 / Other relevant environment information:

附加信息 / Additional Information 添加与问题相关的任何其他信息 / Add any other information related to the issue.

F:\Workspace\PycharmProjects\Langchain-Chatchat>python init_database.py --recreate-vs recreating all vector stores 2024-02-28 00:17:34,622 - faiss_cache.py[line:92] - INFO: loading vector store in 'samples/vector_store/chatglm2-6b' from disk. AttributeError: 'NoneType' object has no attribute 'conjugate'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "F:\Workspace\PycharmProjects\Langchain-Chatchat\init_database.py", line 107, in folder2db(kb_names=args.kb_name, mode="recreate_vs", embed_model=args.embed_model) File "F:\Workspace\PycharmProjects\Langchain-Chatchat\server\knowledge_base\migrate.py", line 121, in folder2db kb.create_kb() File "F:\Workspace\PycharmProjects\Langchain-Chatchat\server\knowledge_base\kb_service\base.py", line 81, in create_kb self.do_create_kb() File "F:\Workspace\PycharmProjects\Langchain-Chatchat\server\knowledge_base\kb_service\faiss_kb_service.py", line 51, in do_create_kb self.load_vector_store() File "F:\Workspace\PycharmProjects\Langchain-Chatchat\server\knowledge_base\kb_service\faiss_kb_service.py", line 28, in load_vector_store return kb_faiss_pool.load_vector_store(kb_name=self.kb_name, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\Workspace\PycharmProjects\Langchain-Chatchat\server\knowledge_base\kb_cache\faiss_cache.py", line 102, in load_vector_store vector_store = self.new_vector_store(embed_model=embed_model, embed_device=embed_device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\Workspace\PycharmProjects\Langchain-Chatchat\server\knowledge_base\kb_cache\faiss_cache.py", line 60, in new_vector_store vector_store = FAISS.from_documents([doc], embeddings, normalize_L2=True,distance_strategy="METRIC_INNER_PRODUCT") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\Python\python311\Lib\site-packages\langchain_core\vectorstores.py", line 508, in from_documents return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\Python\python311\Lib\site-packages\langchain_community\vectorstores\faiss.py", line 959, in from_texts embeddings = embedding.embed_documents(texts) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\Workspace\PycharmProjects\Langchain-Chatchat\server\knowledge_base\kb_service\base.py", line 425, in embed_documents return normalize(embeddings).tolist() ^^^^^^^^^^^^^^^^^^^^^ File "F:\Workspace\PycharmProjects\Langchain-Chatchat\server\knowledge_base\kb_service\base.py", line 37, in normalize norm = np.linalg.norm(embeddings, axis=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<array_function internals>", line 200, in norm File "F:\Python\python311\Lib\site-packages\numpy\linalg\linalg.py", line 2541, in norm s = (x.conj() * x).real ^^^^^^^^ TypeError: loop of ufunc does not support argument 0 of type NoneType which has no callable conjugate method

[dosubot[bot]]

🤖

你好，@YanzyUpEveryDay！我是Dosu，一个旨在帮助解决bug、回答问题，并协助你成为贡献者的机器人。在等待人类维护者的同时，我在这里为你提供帮助。我会立即查看你的问题，并尽快给出完整的回答。

[dosubot[bot]]

🤖

The error you're encountering, AttributeError: 'NoneType' object has no attribute 'conjugate', typically occurs when a method is called on a None object. In this case, it seems to be happening when the normalize function is trying to calculate the norm of the embeddings using np.linalg.norm(embeddings, axis=1). This suggests that embeddings is None at the time of this call.

Based on the traceback you provided, it seems that the embeddings are supposed to be generated by the embed_documents function in the base.py file. If embeddings is None, it could mean that this function is not returning the expected output.

Here are some steps you can take to troubleshoot this issue:

1. Check the embed_documents function: Make sure that this function is correctly generating the embeddings. You can add some debug prints in this function to check the output.

2. Check the input to embed_documents function: The input to this function seems to be texts. Make sure that texts is not None or empty when it's passed to embed_documents.

3. Check the embedding model: You mentioned that you're using the moka-ai/m3e-base model for embeddings. Make sure that this model is correctly loaded and working as expected. You can test this by trying to generate embeddings for some sample texts.

4. Check the version of numpy and other dependencies: Sometimes, such issues can be caused by incompatibility between different versions of dependencies. Make sure that you're using the versions of numpy and other dependencies that are compatible with Langchain-Chatchat v2.0.1.

If none of the above steps help, it would be helpful if you could provide more details about your setup and the steps you're following. This would help in identifying the root cause of the issue.

In addition, I found some similar issues in the Langchain-Chatchat repository that were resolved by following certain steps. You can refer to these issues for potential solutions:

1. [BUG] 简洁阐述问题 / Concise description of the issue
2. python init_database.py --recreate-vs提示CUDA没有启用
3. [BUG] 3. 初始化知识库和配置文件 报错 python init_database.py --recreate-vs

I hope this helps! Let me know if you have any other questions.

Sources

• [BUG] 简洁阐述问题 / Concise description of the issue
• python init_database.py --recreate-vs提示CUDA没有启用
• [BUG] 3. 初始化知识库和配置文件 报错 python init_database.py --recreate-vs
• init_database.py
• server/knowledge_base/kb_service/base.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[zRzRzRzRzRzRzR]

检查模型完整性