feat: support TEI embedding service, configurable reranking model

[KKenny0]

Description

• Add support for TEI (Text-Embedding-Inference) API compatible embedding/reranking service.
• Separate Reranking model into configurable module. (You can see the changes in the Tab Resources-Rerankings)

Type of change

• [x] New features (non-breaking change).
• [ ] Bug fix (non-breaking change).
• [ ] Breaking change (fix or feature that would cause existing functionality not to work as expected).

Checklist

• [x] I have performed a self-review of my code.
• [x] I have performed tests.
• [x] There is a reference to the original bug report and related work.
• [x] I have commented on my code, particularly in hard-to-understand areas.
• [x] The feature is well documented.

[taprosoft]

Thanks for the contribution @KKenny0. Can you take a look at the CI failures and fix it?

[KKenny0]

Thanks for the contribution @KKenny0. Can you take a look at the CI failures and fix it?

Thanks for the review. I've looked into the CI failures and have made the necessary fixes. The updated changes should resolve the issues.

[sunnf8888]

xinference launch --model-name jina-reranker-v2 --model-type rerank --endpoint "http://127.0.0.1:6000" 我用xinference 启动，如何配置呢 vendors 选择哪一个在CohereReranking 和TeiFastReranking？

[sunnf8888]

xinference launch --model-name jina-reranker-v2 --model-type rerank --endpoint " http://127.0.0.1:6000/ " I started it with xinference, how do I configure it Which do vendors choose between CohereReranking and TeiFastReranking? How to configure the specification?

[KKenny0]

xinference launch --model-name jina-reranker-v2 --model-type rerank --endpoint "http://127.0.0.1:6000" 我用xinference 启动，如何配置呢 vendors 选择哪一个在CohereReranking 和TeiFastReranking？

Cohere 目前只支持官方的 api key 访问。 xinference 的 cURL 请求体结构 Tei 的不兼容，一个可实施的方案是，参考kotaemon/rerankings/tei_fast_rerank.py，自己新增对于 xinference 的支持，然后在 ktem/rerankings/manager.py 中新增 xinference 的入口即可。

有疑问欢迎沟通。

[sunnf8888]

多谢， 我研究一下，您提供的方式。

能否提供，微信等的联系方式，可以付费咨询等。

烈日 @.***

 

------------------ 原始邮件 ------------------ 发件人: "Cinnamon/kotaemon" @.>; 发送时间: 2024年9月29日(星期天) 中午1:15 @.>; @.@.>; 主题: Re: [Cinnamon/kotaemon] feat: support TEI embedding service, configurable reranking model (PR #287)

xinference launch --model-name jina-reranker-v2 --model-type rerank --endpoint "http://127.0.0.1:6000" 我用xinference 启动，如何配置呢 vendors 选择哪一个在CohereReranking 和TeiFastReranking？

Cohere 目前只支持官方的 api key 访问。 xinference 的 cURL 请求体结构 Tei 的不兼容，一个可实施的方案是，参考kotaemon/rerankings/tei_fast_rerank.py，自己新增对于 xinference 的支持，然后在 ktem/rerankings/manager.py 中新增 xinference 的入口即可。

有疑问欢迎沟通。

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[KKenny0]

你可以将联系方式单独 email 给我，我到时加你 :)。

From: sunnf8888 @.> Date: Sunday, September 29, 2024 at 15:06 To: Cinnamon/kotaemon @.> Cc: KennyWu @.>, Mention @.> Subject: Re: [Cinnamon/kotaemon] feat: support TEI embedding service, configurable reranking model (PR #287) 多谢， 我研究一下，您提供的方式。

能否提供，微信等的联系方式，可以付费咨询等。

烈日 @.***

 

------------------ 原始邮件 ------------------ 发件人: "Cinnamon/kotaemon" @.>; 发送时间: 2024年9月29日(星期天) 中午1:15 @.>; @.@.>; 主题: Re: [Cinnamon/kotaemon] feat: support TEI embedding service, configurable reranking model (PR #287)

xinference launch --model-name jina-reranker-v2 --model-type rerank --endpoint "http://127.0.0.1:6000" 我用xinference 启动，如何配置呢 vendors 选择哪一个在CohereReranking 和TeiFastReranking？

Cohere 目前只支持官方的 api key 访问。 xinference 的 cURL 请求体结构 Tei 的不兼容，一个可实施的方案是，参考kotaemon/rerankings/tei_fast_rerank.py，自己新增对于 xinference 的支持，然后在 ktem/rerankings/manager.py 中新增 xinference 的入口即可。

有疑问欢迎沟通。

― Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

― Reply to this email directly, view it on GitHubhttps://github.com/Cinnamon/kotaemon/pull/287#issuecomment-2381139043, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEJWIGLBUNYBKQTAQUECJPDZY6RIZAVCNFSM6AAAAABOGMZWVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBRGEZTSMBUGM. You are receiving this because you were mentioned.Message ID: @.***>

[sunnf8888]

我的电话 13946091831

微信也是这个。

烈日 @.***

 

------------------ 原始邮件 ------------------ 发件人: "Cinnamon/kotaemon" @.>; 发送时间: 2024年9月29日(星期天) 下午3:13 @.>; @.@.>; 主题: Re: [Cinnamon/kotaemon] feat: support TEI embedding service, configurable reranking model (PR #287)

你可以将联系方式单独 email 给我，我到时加你 :)。

From: sunnf8888 @.> Date: Sunday, September 29, 2024 at 15:06 To: Cinnamon/kotaemon @.> Cc: KennyWu @.>, Mention @.> Subject: Re: [Cinnamon/kotaemon] feat: support TEI embedding service, configurable reranking model (PR #287) 多谢， 我研究一下，您提供的方式。

能否提供，微信等的联系方式，可以付费咨询等。

烈日 @.***

 

------------------ 原始邮件 ------------------ 发件人: "Cinnamon/kotaemon" @.>; 发送时间: 2024年9月29日(星期天) 中午1:15 @.>; @.@.>; 主题: Re: [Cinnamon/kotaemon] feat: support TEI embedding service, configurable reranking model (PR #287)

xinference launch --model-name jina-reranker-v2 --model-type rerank --endpoint "http://127.0.0.1:6000" 我用xinference 启动，如何配置呢 vendors 选择哪一个在CohereReranking 和TeiFastReranking？

Cohere 目前只支持官方的 api key 访问。 xinference 的 cURL 请求体结构 Tei 的不兼容，一个可实施的方案是，参考kotaemon/rerankings/tei_fast_rerank.py，自己新增对于 xinference 的支持，然后在 ktem/rerankings/manager.py 中新增 xinference 的入口即可。

有疑问欢迎沟通。

― Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

― Reply to this email directly, view it on GitHub<https://github.com/Cinnamon/kotaemon/pull/287#issuecomment-2381139043>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AEJWIGLBUNYBKQTAQUECJPDZY6RIZAVCNFSM6AAAAABOGMZWVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBRGEZTSMBUGM>. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[taprosoft]

Thanks @KKenny0 for your great contribution. Just fixed the logic a bit and added default configuration for CohereRerank in flowsettings.

[KKenny0]

Thanks @KKenny0 for your great contribution. Just fixed the logic a bit and added default configuration for CohereRerank in flowsettings.

Thank you for your kind words. I'm glad my contribution was useful. It's great to see the improvements you've made. Let's keep making the project better together.