ragas icon indicating copy to clipboard operation
ragas copied to clipboard

Context Precision Metric Binary values

Open kaan9700 opened this issue 5 months ago • 1 comments

I have noticed that if you look at the evaluated data set, the metric is determined individually for each data point of the dataset. The output of the console is then the mean value of all calculated metrics. With Context Precision, I have noticed that the individual values (values per data point) are always binary, i.e. 0 or 1. Is this normal? Is there only this range of values for the individual data points, or have I made a mistake?

The whole thing can be reproduced if you use the small test set from the RAGAS documentation and save the generated dataframe as a CSV.

kaan9700 avatar Sep 15 '24 21:09 kaan9700