ragas
ragas copied to clipboard
Context Precision Metric Binary values
I have noticed that if you look at the evaluated data set, the metric is determined individually for each data point of the dataset. The output of the console is then the mean value of all calculated metrics. With Context Precision, I have noticed that the individual values (values per data point) are always binary, i.e. 0 or 1. Is this normal? Is there only this range of values for the individual data points, or have I made a mistake?
The whole thing can be reproduced if you use the small test set from the RAGAS documentation and save the generated dataframe as a CSV.