DSI-QG

DSI-QG copied to clipboard

About the data split of NQ 320K

22

Hi, Dr. Zhuang. Thanks for your contribution again. I've successfully conducted some experiments on MSMARCO with DSI-QG. To keep going, I plan to conduct the experiments on NQ dataset. The...

hi-i-m-GTooth

How to calculate the metric HITS

1

Hi, I am a bit confused on how the metric HITS is calculated. According to the article, HITS is defined as "the proportion of the correct docids ranked in the...

oussaidene

cannt use huggingface datasets and model in online way

4

bencaocs

Reproduce hit@10 is zero

2

Hi，I want to evaluate the DSI performance, use the command ,  , and train 5000steps ,hits@10 is zero,Could you push the right evaluation result.Thank you !

bit-pku-zdf

Regarding the "cross-encoder ranker" mentioned in the paper

1

In the paper, a "cross-encoder ranker" is used to rank the queries. Could you point me to the code of the ranking in this repo? Thanks!

yubad2000

Single GPU feasibility

1

Hi. Note that the demo command will launch the training in 8 GPU. Have you tested running this task on a single GPU, and how long will it take? I...

gusye1234

Data Files for NQ

3

Hi @ArvinZhuang, Thanks for your great efforts! I was wondering if it is possible to provide the data files for the NQ dataset? Thanks!

Gouzi3618

Train DSI-QG model

1

Step 1 of readme, description of script saying query generation. But in run.py, "DocTqueryTrainer" use "IndexingTrainDataset" and it make document/query to docid dataset. So, result of training model make just...

hibiki12y

Predictions for individual queries

1

It seems that the code will output metrics. I wonder if there is a way to output the actual predictions per query?

hlncrg

NQ320k preprocessing?

4

Hi, I tried to reproduce the results of your experiments on NQ320k dataset as per the table from your paper  To do this, I referred to [your script](https://github.com/ArvinZhuang/DSI-transformers/blob/main/data/NQ/create_NQ_train_vali.py) from...

kisozinov