dify add Ollama rerank model

Self Checks

[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Hi, I am trying to use Ollama only with Dify, and I am missing rerank models in the usecase of RAG. Could you please add rerank models for Ollama, besides LLM and text embedding.

2. Additional context or comments

No response

3. Can you help us with this feature?

[X] I am interested in contributing to this feature.

Nov 20 '24 22:11 shenhai-ran

I do not see rerank type here https://www.ollama.com/search.

Nov 21 '24 01:11 crazywoola

@crazywoola yes, you are absolutely right. But now Ollama supports models from Huggingface, as in this blog https://huggingface.co/docs/hub/en/ollama, and there are rerank model of GGUF format on Huggingface, such as https://huggingface.co/gpustack/bge-reranker-v2-m3-GGUF.

Do you think Dify can support that?

Nov 21 '24 08:11 shenhai-ran

Urgently needed, currently in the Dify configuration, the reranker type is not supported, which forces me to choose another model supplier.

Dec 14 '24 07:12 dzgdp888

@crazywoola try this https://ollama.com/linux6200/bge-reranker-v2-m3

Dec 14 '24 07:12 dzgdp888

@crazywoola try this https://ollama.com/linux6200/bge-reranker-v2-m3

Where should I set this? I only can set LLM and embedding model in ollama

Dec 23 '24 05:12 githubjason-18

Hi, @shenhai-ran. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

You requested the addition of rerank models for the Ollama use case, particularly for RAG scenarios.
Crazywoola noted the absence of a rerank type on the Ollama search page.
You mentioned that Ollama now supports rerank models from Huggingface in GGUF format.
Dzgdp888 highlighted an urgent need for this feature and suggested an alternative model supplier.
Githubjason-18 asked for guidance on setting up the reranker model, indicating a need for further configuration instructions.

Next Steps:

Please let me know if this issue is still relevant to the latest version of the Dify repository by commenting here.
If there is no further activity, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

Jan 23 '25 16:01 dosubot[bot]

pls add it

Feb 12 '25 08:02 nzlov

pls add it

+1

Feb 19 '25 01:02 rowstop

pls add it

+1

Feb 19 '25 02:02 Pandaaaa906

+1

pls add it

Feb 19 '25 10:02 nvictorm

+1

Feb 20 '25 10:02 leemeng0x61

+1

Feb 21 '25 05:02 cshs20120831

+1 please.

Feb 24 '25 02:02 youyouzh

+1

Feb 26 '25 06:02 bobo521bobo

+1, and why is it not planned

Feb 26 '25 06:02 AlgorithmicKing737

pls add it

+10086

Feb 26 '25 08:02 blackcatcjy

+1

Feb 27 '25 00:02 cwyalpha

+1

Feb 28 '25 02:02 liguochuan00

Hey guys, we should wait for ollama/ollama#7219 to be merged

Feb 28 '25 07:02 rowstop

give us the ollama rerank support!!!! I need to install a massive xinference to provide the ReRank model, which is really foolish. It's obvious that Olama provides the ReRank model but i cant use it