RocketQA icon indicating copy to clipboard operation
RocketQA copied to clipboard

big difference between setting title and not setting title

Open Jeffwan opened this issue 2 years ago • 1 comments

We are using cross decoder to rerank the results. for some QA pairs. We use following format to get the ranking scores.

<query, para1, title1>
<query, para2, title2>
<query, para3, title3>
model.matching(query: List[str], para: List[str], title: List[str])

There's one observation I notice is, for the same question, setting title give me much better results, top1 or top2. However, If I do not set title, it is really bad (~top10 for 20 records).

I am curious why title makes a big difference here? In my case, for query, we have many tiles with same name. I felt adding title is not a big deal but it is

Jeffwan avatar Jun 01 '23 06:06 Jeffwan

https://github.com/PaddlePaddle/RocketQA/blob/1746b938d659c7f8d0b9f960e3199dcbd945adac/rocketqa/encoder/cross_encoder.py#L160-L166

Jeffwan avatar Jun 01 '23 06:06 Jeffwan