continuous-eval icon indicating copy to clipboard operation
continuous-eval copied to clipboard
verified publisher icon relari-ai

Metadata comparison for retrieval metrics

Open yisz opened this issue 1 year ago • 1 comments

yisz avatar Apr 01 '24 22:04 yisz

@kelvinchanwh I actually had this issue open a while ago. I will find some time to implement it. I plan to do the below. Let me know if you have any suggestions.

TODO: Create a new metric PrecisionRecallF1_Metadata that compares the metadata of retrieved_context vs. metadata of ground_truth_context. metadata should be a json containing information such as can be document id / page number / etc.

yisz avatar Jun 14 '24 16:06 yisz