Knowledge created in versions prior to 1.9.1 is not usable after upgrading to 1.9.2

[kurokobo]

Self Checks

• [x] I have read the Contributing Guide and Language Policy.
• [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report, otherwise it will be closed.
• [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• [x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.9.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

1. Deploy fresh 1.9.1 (or older) of Dify with default VDB (Weaviate 1.19.0)
2. Create Knowledge and upload document
3. Ensure Retreival Testing works
4. Upgrade Dify to 1.9.2
5. Try Retreival Testing in the existing Knowledge

✔️ Expected Behavior

Retreival Testing works.

❌ Actual Behavior

Query call with protocol GRPC search failed with message extract target vectors: class Vector_index_b7ff402a_fea1_44f5_bed3_c26f5e25a369_Node does not have named vector default configured. Available named vectors map[].

[dosubot[bot]]

This is a known issue when upgrading from Dify 1.9.1 (or earlier) to 1.9.2 with Weaviate as the vector database. The error occurs because older knowledge base collections lack the required named vector configuration (default) that Dify 1.9.2 expects for retrieval operations. The upgrade process does not automatically migrate or patch existing Weaviate collections to add this configuration, so retrieval fails for knowledge created before the upgrade [source].

To restore retrieval functionality, you need to re-index or re-upload the affected knowledge/documents so that the collections are recreated with the correct named vector configuration. There is currently no automated migration script in the codebase to patch legacy collections; manual re-indexing is required [source].

If you have a large amount of legacy knowledge, consider scripting the re-upload process or, if possible, exporting and re-importing the data to trigger re-indexing with the new schema.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[kurokobo]

@DhruvGorasiya @crazywoola Since upgrading Weaviate has been a long-standing issue, this enhancement was highly anticipated, and I’m really grateful for all your hard work. On the other hand, I’m concerned that this might cause significant confusion.

I assume you tested whether “newly created knowledge” works, but did you also test in advance whether “existing knowledge” continues to work?

I’ve already reviewed the documentation about migrating Weaviate. While there are appealing descriptions such as the following, unfortunately, in practice the Automatic Migration didn’t allow us to use knowledge created before the upgrade.

Path A: Docker Compose Users (Recommended)
If you’re using Dify’s standard Docker Compose setup, the Weaviate version is automatically updated when you upgrade Dify. No manual configuration is required.

Data Migration for Legacy Versions
Automatic Migration
In most cases, Weaviate 1.27.0 will automatically migrate data from 1.19.0:

Therefore, users need to follow the Manual Migration steps, but even then there are issues:

• The backup-filesystem module isn’t loaded in Weaviate by default, so we can’t create a backup in the first place
• By default, localhost:8080 isn’t exposed, so the suggested commands in docs don’t work as written. Also, curl is not available in Weaviate container.
• There’s no mention of the required Authorization header either

Here’s what I think should be done—what are your thoughts?

• Implement some automatic migration feature to migrate existing knowledges

Or,

• Clearly state that manual data migration for Weaviate is required when upgrading Dify
• The procedure should be: (EDIT: This might not be entirely accurate. See my follow-up comments.)
  • Add ENABLE_MODULES and BACKUP_FILESYSTEM_PATH configurations to the old version of Weaviate (so that backups are saved to a directory mounted via volumes)
  • Trigger the backup from the worker container (this is straightforward since WEAVIATE_ENDPOINT and WEAVIATE_API_KEY environment variables are available)
  • Upgrade Dify to 1.9.2
  • Add ENABLE_MODULES and BACKUP_FILESYSTEM_PATH configurations to the new version of Weaviate (so you can restore from the mounted backup directory)
  • Trigger the restore from the worker container

[kurokobo]

• The procedure should be:
  • Add ENABLE_MODULES and BACKUP_FILESYSTEM_PATH configurations to the old version of Weaviate (so that backups are saved to a directory mounted via volumes)
  • Trigger the backup from the worker container (this is straightforward since WEAVIATE_ENDPOINT and WEAVIATE_API_KEY environment variables are available)
  • Upgrade Dify to 1.9.2
  • Add ENABLE_MODULES and BACKUP_FILESYSTEM_PATH configurations to the new version of Weaviate (so you can restore from the mounted backup directory)
  • Trigger the restore from the worker container

Since this was just an idea I wrote before thoroughly testing it to follow Manual Migration, I actually tried it out, but the error wasn't resolved after all 🤔

I'm not a Weaviate expert, so I'm not sure about the details, but it might be related to changes in api/core/rag/datasource/vdb/weaviate/weaviate_vector.py ?

[jtfeng]

need help，same problem！

[Tabll]

it feels like every upgrade has some issues. last time it was v2.0.0-beta.2 -> v1.9.1 https://github.com/langgenius/dify/issues/26091 fortunately, I checked this issues before upgrading. 😒😒

[a397038588-jpg]

I think this is a very serious issue. It has caused the already created knowledge base to not function properly, and the documentation doesn't mention how to solve it.

[ChiuHsin]

same problem

[HZJprince]

我怀疑他们每次更新都不测试过功能就发布了，每次更新都各种不能用

[Gum97]

same problem, How to fix ???????

[lema-founders]

@crazywoola any updates on this?

[DhruvGorasiya]

Update: Migration Solution Available

We've identified the root cause and developed a migration solution.

The Issue

Weaviate 1.27.0+ requires a new schema format with named vectors (vectorConfig). Existing data from Weaviate 1.19.0 uses the old format, causing the error when querying with weaviate-client v4.

The Solution

We're preparing:

• Migration script that preserves all embeddings and data
• Step-by-step guide for Docker and Cloud users
• Automated detection for multiple knowledge bases

Timeline

Migration guide and scripts will be published by the end of the day today to by tomorrow morning latest.

Temporary Fix

Downgrade to Dify 1.9.1 if you need immediate access to your knowledge bases:

git checkout v1.9.1
cd docker
docker compose --profile weaviate up -d

Updates coming soon.

[aijuntao]

My upgrade from 1.9.1 to 1.9.2 also had this issue. In order to rebuild the index, I changed the Embedding 模型 of the knowledge base. At this time, the knowledge base will automatically rebuild the index, and the problem has been solved. Now the functions can be used normally

[Tabll]

我觉得 + 0.0.1 这种小版本号的升级不适合搞这种不兼容的变动吧，至少也是要 + 0.1.0
+ 0.0.1 应该就只包含普通的bug修复

[nadazero]

My upgrade from 1.9.1 to 1.9.2 also had this issue. In order to rebuild the index, I changed the Embedding 模型 of the knowledge base. At this time, the knowledge base will automatically rebuild the index, and the problem has been solved. Now the functions can be used normally

不失为一个应急的workround办法，但是还是有问题。 1.需要切换Embedding 模型，如果只装了1个模型就没法切换了。 2.需评估切换后Embedding 模型的效果是否符合预期。Embedding 模型也是有区别的。 3.耗时问题，如果知识库里有大量文档，reindex需要多久？相当于停止服务一段时间了。

[aijuntao]

My upgrade from 1.9.1 to 1.9.2 also had this issue. In order to rebuild the index, I changed the Embedding 模型 of the knowledge base. At this time, the knowledge base will automatically rebuild the index, and the problem has been solved. Now the functions can be used normally

不失为一个应急的workround办法，但是还是有问题。 1.需要切换Embedding 模型，如果只装了1个模型就没法切换了。 2.需评估切换后Embedding 模型的效果是否符合预期。Embedding 模型也是有区别的。 3.耗时问题，如果知识库里有大量文档，reindex需要多久？相当于停止服务一段时间了。

是的为了快速重建索引，我们临时加了个 Embedding 模型，重建完成后，又换回原来的模型

[KickUasS]

My upgrade from 1.9.1 to 1.9.2 also had this issue. In order to rebuild the index, I changed the Embedding 模型 of the knowledge base. At this time, the knowledge base will automatically rebuild the index, and the problem has been solved. Now the functions can be used normally

不失为一个应急的workround办法，但是还是有问题。 1.需要切换Embedding 模型，如果只装了1个模型就没法切换了。 2.需评估切换后Embedding 模型的效果是否符合预期。Embedding 模型也是有区别的。 3.耗时问题，如果知识库里有大量文档，reindex需要多久？相当于停止服务一段时间了。

是的为了快速重建索引，我们临时加了个 Embedding 模型，重建完成后，又换回原来的模型

直接在知识库的设置里面换嵌入模型，会自动重建所有文档的索引吗？

[yasu-oh]

Hi @DhruvGorasiya , thank you for the update. After upgrading Dify to v1.9.2, I followed the official conversion steps to migrate Weaviate from 1.19.0 to 1.27.0, but now the knowledge base is no longer accessible. https://github.com/langgenius/dify/blob/main/docs/weaviate/WEAVIATE_MIGRATION_GUIDE/README.md

You mentioned that “Migration guide and scripts will be published by the end of the day today to by tomorrow morning latest”, but I couldn’t find them anywhere. Could you please share where those migration guides or scripts were published?

Thanks in advance for your help!

[xiaoyesoso]

My upgrade from 1.9.1 to 1.9.2 also had this issue. In order to rebuild the index, I changed the Embedding 模型 of the knowledge base. At this time, the knowledge base will automatically rebuild the index, and the problem has been solved. Now the functions can be used normally我从 1.9.1 升级到 1.9.2 也出现了这个问题。为了重建索引，我更改了知识库的 Embedding 模型。此时知识库会自动重建索引，问题已解决。现在功能可以正常使用了。

Is there an official solution to directly rebuild the index?

[aijuntao]

My upgrade from 1.9.1 to 1.9.2 also had this issue. In order to rebuild the index, I changed the Embedding 模型 of the knowledge base. At this time, the knowledge base will automatically rebuild the index, and the problem has been solved. Now the functions can be used normally我从 1.9.1 升级到 1.9.2 也出现了这个问题。为了重建索引，我更改了知识库的 Embedding 模型。此时知识库会自动重建索引，问题已解决。现在功能可以正常使用了。

Is there an official solution to directly rebuild the index?

没有官方回复吧？官方指导文档，我的理解是重新创建知识库。

[aijuntao]

Hi @DhruvGorasiya , thank you for the update. After upgrading Dify to v1.9.2, I followed the official conversion steps to migrate Weaviate from 1.19.0 to 1.27.0, but now the knowledge base is no longer accessible. https://github.com/langgenius/dify/blob/main/docs/weaviate/WEAVIATE_MIGRATION_GUIDE/README.md

You mentioned that “Migration guide and scripts will be published by the end of the day today to by tomorrow morning latest”, but I couldn’t find them anywhere. Could you please share where those migration guides or scripts were published?

Thanks in advance for your help!

求人不如求己呀。我家的 Weaviate 从1.19 一个版本一个版本的升级，从 1.19->1.20->1.21~1.27 因为 Weaviate 有几个版本不能跳过，安全起见，一个版本一个版本升级到目标版本吧

[aijuntao]

Hi @DhruvGorasiya , thank you for the update. After upgrading Dify to v1.9.2, I followed the official conversion steps to migrate Weaviate from 1.19.0 to 1.27.0, but now the knowledge base is no longer accessible. https://github.com/langgenius/dify/blob/main/docs/weaviate/WEAVIATE_MIGRATION_GUIDE/README.md You mentioned that “Migration guide and scripts will be published by the end of the day today to by tomorrow morning latest”, but I couldn’t find them anywhere. Could you please share where those migration guides or scripts were published? Thanks in advance for your help!

求人不如求己呀。我家的 Weaviate 从1.19 一个版本一个版本的升级，从 1.19->1.20->1.21~1.27 因为 Weaviate 有几个版本不能跳过，安全起见，一个版本一个版本升级到目标版本吧

如果是 Docker compose 部署，Dify 1.9.2 版本的 docker 镜像重新打包，Weaviate 升级完后，建议把 1.9.2 的镜像都删除，重新下载，能解决不少问题

[yasu-oh]

Hi @laipz8200,

Could you please confirm whether you’re aware of this issue? If this behavior is already recognized, it might be helpful to include a note or warning in the release notes to alert users about the potential impact.

Thanks for your attention to this!

[KickUasS]

Hi @laipz8200,

Could you please confirm whether you’re aware of this issue? If this behavior is already recognized, it might be helpful to include a note or warning in the release notes to alert users about the potential impact.

Thanks for your attention to this!

Hi @DhruvGorasiya , thank you for the update. After upgrading Dify to v1.9.2, I followed the official conversion steps to migrate Weaviate from 1.19.0 to 1.27.0, but now the knowledge base is no longer accessible. https://github.com/langgenius/dify/blob/main/docs/weaviate/WEAVIATE_MIGRATION_GUIDE/README.md You mentioned that “Migration guide and scripts will be published by the end of the day today to by tomorrow morning latest”, but I couldn’t find them anywhere. Could you please share where those migration guides or scripts were published? Thanks in advance for your help!

求人不如求己呀。我家的 Weaviate 从1.19 一个版本一个版本的升级，从 1.19->1.20->1.21~1.27 因为 Weaviate 有几个版本不能跳过，安全起见，一个版本一个版本升级到目标版本吧

如果是 Docker compose 部署，Dify 1.9.2 版本的 docker 镜像重新打包，Weaviate 升级完后，建议把 1.9.2 的镜像都删除，重新下载，能解决不少问题

这次的更新，换就坏在这里，用docker-compose跟着1.9.1升级到1.9.2，weaviate版本会直接从1.19.0跳到1.27.0，weaviate官方是完全禁止这样的升级的行为的！

[KickUasS]

Hi @laipz8200,

Could you please confirm whether you’re aware of this issue? If this behavior is already recognized, it might be helpful to include a note or warning in the release notes to alert users about the potential impact.

Thanks for your attention to this!

The Dify team did add a 'warning' to the 1.9.2 update notes, but only after this issue was widely reported. If you were lucky enough to have a full backup from before the update, you could downgrade to save your existing knowledge. But for those like me, who backed up after launching the new version, the only option is to re-upload and re-index every single document!

[DhruvGorasiya]

@KickUasS You can try this quick workaround: Go to your Knowledge Base settings, switch the embedding model to a different one, and hit Save, that should automatically trigger re-indexing.

If you want to revert to your original model, just repeat the process: select another model, save it, and then switch back to your original one.

This is only a temporary fix, we’re currently working on a proper migration script to handle data updates without needing a full re-index.

[jtfeng]

I re-pulled the v1.9.2 image and found that the bug has been fixed.

[cheatofrom]

我重新拉取了v1.9.2镜像，发现bug已经修复。

no，today，we update the v1.9.2。the question is still。we also delete the local image and Re-pull 。

[nadazero]

谁能说下新的1.9.2镜像修改了什么内容？

[cheatofrom]

@KickUasS您可以尝试这个快速解决方法： 转到您的知识库设置，将嵌入模型切换到另一个模型，然后点击保存，这应该会自动触发重新索引。

如果您想恢复到原始模型，只需重复此过程：选择另一个模型，保存它，然后切换回原始模型。

这只是一个临时修复，我们目前正在研究适当的迁移脚本来处理数据更新，而无需完全重新索引。

This is useful

[HZJprince]

我重新拉取了v1.9.2镜像，发现bug已经修复。

no，today，we update the v1.9.2。the question is still。we also delete the local image and Re-pull 。

他们的镜像是不会更新的，除非发下一个版本时，才会把最近修复的bugs更新到镜像里面去，所以1.9.2的镜像还是原来那些问题，除非你自己拉代码下来重新build镜像，不想自动动手就等下一版本吧