dify dify 1.0.1 integration with external knowledge base ragflow recall is invalid.

Self Checks

[x] This is only for bug report, if you would like to ask a question, please head to Discussions.
[x] I have searched for existing issues search for existing issues, including closed ones.
[x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.0.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

After adding the external ragflow knowledge base, performing a recall test returned no results. However, recall on ragflow itself yields results.

✔️ Expected Behavior

Normally, recall data is obtained from ragflow.

❌ Actual Behavior

No response

Mar 16 '25 05:03 mfjiang

I found some similar discussions that might be relevant to your issue:

The api cannot retrieve the external knowledge base - This discussion is still open and might provide insights into similar issues with external knowledge base retrieval ^[1].
Do you have the same bug when I use ragflow as a external knowledge - Another open discussion that addresses issues when using RAGFlow as an external knowledge base ^[2].
When I was using the case of workflow+knowledge base, I encountered a problem - This discussion is also open and might contain relevant information about workflow and knowledge base integration issues ^[3].

Regarding your issue with Dify 1.0.1, there are a few potential reasons why the recall test might return no results:

Ensure that the external knowledge base API response follows the required JSON structure, which must include a 'records' array with 'metadata', 'score', 'title', and 'content' fields. If the response doesn't match this structure, no data will be displayed even if the HTTP status code is successful ^[4].
There was a known bug in Dify version 0.15.3 where the rerank model switch in the 'Recall Settings' would automatically turn on, causing the knowledge retrieval module to return empty results. This bug has been fixed in a recent update, so ensure you are using the latest version of Dify to avoid this issue ^[5].
If you are using multiple knowledge bases, an existing knowledge base might stop working after creating a new one in another chat flow. Try renaming the affected knowledge base, as this has been shown to restore its functionality ^[6].
In Dify 1.0.0, there is a known bug where agents cannot read from knowledge bases, resulting in a PluginInvokeError, particularly affecting keyword recall functionality. This issue is present in the self-hosted Docker version and will be addressed in an upcoming release ^[7].

These steps might help you troubleshoot and resolve the issue with your external knowledge base integration.

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

Mar 16 '25 05:03 dosubot[bot]

I have the same question too . I am using dify 1.1.0 now. You can even successfully add the ragdflow API.
Monitor the monitoring log of docker ragflow, it's ok.
by using docker logs -f dify_docker-api-1，it shows that the api is forbidden.

It's so confusing.

Mar 19 '25 03:03 bruceOrange22

There are several reasons for the problem:

The port should be 9380 (the proxy method of port 81 is incorrect)
Through tcpdump packet capture, it is found that dify and ragflow have very different parameter formats for defining requests and responses. For specific comparisons: https://docs.dify.ai/zh-hans/guides/knowledge-base/external-knowledge-api-documentation, https://ragflow.io/docs/dev/http_api_reference#retrieve-chunks, you can see that there are obvious differences between the two

My solution: Add ragflow-gateway, use nginx+lua to convert requests and responses. Currently, the request is successful, and the response conversion is still being written.

I hope the official can adapt the parameter format of ragflow as soon as possible

Mar 19 '25 09:03 liamashes

There are several reasons for the problem:

The port should be 9380 (the proxy method of port 81 is incorrect)

Through tcpdump packet capture, it is found that dify and ragflow have very different parameter formats for defining requests and responses. For specific comparisons: https://docs.dify.ai/zh-hans/guides/knowledge-base/external-knowledge-api-documentation, https://ragflow.io/docs/dev/http_api_reference#retrieve-chunks, you can see that there are obvious differences between the two

My solution: Add ragflow-gateway, use nginx+lua to convert requests and responses. Currently, the request is successful, and the response conversion is still being written.
I hope the official can adapt the parameter format of ragflow as soon as possible

agree!I change the port to 8080(just not 81,and it is free for my PC),it's ok now.

Mar 19 '25 09:03 bruceOrange22

There are several reasons for the problem:

The port should be 9380 (the proxy method of port 81 is incorrect)

Through tcpdump packet capture, it is found that dify and ragflow have very different parameter formats for defining requests and responses. For specific comparisons: https://docs.dify.ai/zh-hans/guides/knowledge-base/external-knowledge-api-documentation, https://ragflow.io/docs/dev/http_api_reference#retrieve-chunks, you can see that there are obvious differences between the two

My solution: Add ragflow-gateway, use nginx+lua to convert requests and responses. Currently, the request is successful, and the response conversion is still being written. I hope the official can adapt the parameter format of ragflow as soon as possible

agree!I change the port to 8080(just not 81,and it is free for my PC),it's ok now.

Is your knowledge base recallable? Are there any issues with request format differences?

Mar 19 '25 09:03 liamashes

There are several reasons for the problem:

The port should be 9380 (the proxy method of port 81 is incorrect)

Through tcpdump packet capture, it is found that dify and ragflow have very different parameter formats for defining requests and responses. For specific comparisons: https://docs.dify.ai/zh-hans/guides/knowledge-base/external-knowledge-api-documentation, https://ragflow.io/docs/dev/http_api_reference#retrieve-chunks, you can see that there are obvious differences between the two

My solution: Add ragflow-gateway, use nginx+lua to convert requests and responses. Currently, the request is successful, and the response conversion is still being written. I hope the official can adapt the parameter format of ragflow as soon as possible

agree!I change the port to 8080(just not 81,and it is free for my PC),it's ok now.

Is your knowledge base recallable? Are there any issues with request format differences?

after change the port to 8080,it recallable.

However, it is important to note that the correct API must be filled in dify, without spaces or other incorrect characters. I wasted a lot of time here

Mar 28 '25 01:03 bruceOrange22

Hi, @mfjiang. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

Original issue: Bug in Dify v1.0.1 with external ragflow knowledge base failing to return recall results.
I provided troubleshooting steps, including JSON structure checks and updating Dify.
Discussion by @bruceOrange22 and @liamashes identified port configuration and parameter format differences as potential causes.
Resolution involved changing port to 8080 and using ragflow-gateway with nginx+lua for conversion.

Next Steps:

Please confirm if this issue is still relevant to the latest version of Dify by commenting here.
If no updates are provided, the issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

Apr 28 '25 16:04 dosubot[bot]