kotaemon icon indicating copy to clipboard operation
kotaemon copied to clipboard

[BUG] While using complex reasoning method the relevance score is 0.0 always and still it answers the question.

Open arya18mak opened this issue 1 year ago • 2 comments

Description

In complex reasoning method, multi-step retrieval process is being used, so which chunks are getting displayed in the information panel, multi-retrieval meaning different sub-questions and chunks retrieval for different cases. So which of them are selected I kind of explored the code but could'nt understand. And relevance score remains zero why so(the answer is still right,content is highlighted saying full text search, which means correct chunks)

Reproduction steps

1. Go to 'complex setting'
2. Click on 'send/regen'
3. Scroll down to 'information panel and relevance score.'
4. See error

Screenshots

![DESCRIPTION](LINK.png)

Logs

No response

Browsers

No response

OS

Windows

Additional information

No response

arya18mak avatar Oct 07 '24 17:10 arya18mak

I faced the same initially then I just looked at the Retrieval Settings and switched to model that I was using which in my case was ollama and not the default openai that will pull relevance score from underlying LLM and get you what you want.

image

sachinkumaranumula avatar Oct 17 '24 17:10 sachinkumaranumula

I encounter this when using simple or complex. But I'm using openai for everything as I dont have an ollama server to use yet.

jcroy avatar Apr 09 '25 14:04 jcroy