BGE-M3 compute_score function is very inefficient

[nntoan209]

Is there any faster way to perform the function compute_score for BGE-M3 model? According to the code, it will have to encode the whole corpus num_queries times, and if num_queries is large, it will take forever to run.

[staoxiao]

Thanks for your interest in our work! compute_score is an example to compute the hybrid scores. If you have a better implementation, welcome to submit a PR.

If you want to use it in a retrieval system, a better choice is using the compute_score function in re-rank stage, where only a few candidates need to compute scores. You can use sentence embedding to retrieve a small set of candidates, and use compute_score function to re-rank these candidates.