ragas icon indicating copy to clipboard operation
ragas copied to clipboard

[R-258] Question about official docs of Context Precision.

Open JongoPark-complex625 opened this issue 1 year ago • 3 comments

What I understood is that Following case is calculation of Context Precision at K=2.

For that case, Last part's denominator must be 2 not 1.

Is this right?

If i didn't understand well, could you explain more about this?? Following is what i'm thinking.

  1. precision at 1 FP at 1 =1, TP at 1 =0 -> precision at 1 = 0
  2. precision at 2 FP at 2 =1 , TP at 1 =1 -> precision at 2 = 1/2 =0.5 (since we consider top 2 resulte in context chunk items) 3 finally, we have Context Precision at K=2 is (1/2)x(preicision at 1 x indicator at 1 + preicision at 2x indicator at 2) = (1/2)x(0x1+0.5x1)=0.25

thanks.

ragas_docs

R-258

JongoPark-complex625 avatar Apr 24 '24 06:04 JongoPark-complex625

Hi, do you understand? I have this confusion too.

WGS-note avatar May 21 '24 08:05 WGS-note

Hi, do you understand? I have this confusion too.

No.. I'm stuck in same situation.

JongoPark-complex625 avatar May 22 '24 00:05 JongoPark-complex625

hey @WGS-note and @AxelJongoPark, thank you for bringing this up 🙌🏽

@shahules786 could you help them out?

jjmachan avatar May 22 '24 12:05 jjmachan

Metrics were updated as part of v0.2. Please check it out https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/

shahules786 avatar Oct 18 '24 10:10 shahules786