Shitao Xiao
Shitao Xiao
Hi, I run the same command with one GPU (set num_train_epochs=1 for efficiency), and get MRR@10=0.383. Larger num_train_epochs may further improve the result.
Sorry, I forget this issue. For the data format, you can refer to FlagEmbedding: https://github.com/FlagOpen/FlagEmbedding/tree/master/examples We provided some simple examples in it.
Thanks for your interest in RetroMAE! For the enhanced decoder, the collater needs to prepare a mask list for each token. If the number of tokens is large, the data...
Hi, thanks for your interest in our work! Since the Wikipedia dataset has been updated in huggingface, you can use the latest version of data following https://huggingface.co/datasets/wikipedia. To use the...
Code for RetroMAE v2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models
Thanks for your interest in our work! We plan to release the code for RetroMAE v2 in about a week.
Hi, you can use the pipeline in [tevatron](https://github.com/texttron/tevatron/tree/main/examples/coCondenser-nq).
Thanks for your interest in RetroMAE! We fine-tune the model with hard negatives by changing the argument `neg_file` (noted we don't change the hard negatives dynamically in the training process...
For ANCE, we finetune the Shitao/RetroMAE_MSMARCO model. Please use the hyper-parameters in our script, which we found is better.
Thanks! The data is not easy to release due to the information security policy of my current company. I will try to do it. You can train your reranker based...
Thanks for your attention to our work! 1. The loss is not high, and I think it falls within a normal range. 2. If the negative samples are very challenging,...