pygaggle
pygaggle copied to clipboard
[Bug?] uploaded monot5-large-msmarco-10k and monot5-large-msmarco model is exactly the same
Hello, I was trying to evaluate the monot5-large model on the beir benchmark, but I noticed that the uploaded models - 10k and 100k models - has exactly the same model. (when I do diff on terminal, it shows no difference)
Also, the performance of large models is a lot lower than the performance of base models - Could you check if the two uploaded large models are accurate? (maybe relevant to the previous issue)
Thank you!
I ran into the same problem, I tested the 10K and 100K monoT5 on BEIR and found the results were the same, and both were very low, close to zero