Using pygaggle for question answering on a custom dataset

[tanyaroosta]

Hi, I am interested in using pygaggle on a dataset I have for question answering task. However, looking at the examples in the ReadMe file, I am not clear how to do that. Is there an example you can refer me to?

Thanks.

[lintool]

If you're interested in QA, then you can start with our DPR reproduction? https://github.com/castorini/pygaggle/blob/master/docs/experiments-dpr-reader.md

[tanyaroosta]

Thanks for the reply. I had looked at that, and was trying to follow the section titled: "Natural Questions (NQ)", however, I am not clear how the following command needs to change when I have my own set of questions and dataset for answers, i.e. where do you input the set of questions and where does the dataset that contains the answers get passed in:

$ python -um pygaggle.run.evaluate_passage_reader --task wikipedia --retriever score --reader dpr
--settings dpr dprfusion_1.0_0.55
gar garfusion_0.43_0.3053
--retrieval-file data/run.dpr.nq-test.single.bf.json --topk-em 40 80 180 200

On Wed, Jul 21, 2021 at 12:15 PM Jimmy Lin @.***> wrote:

If you're interested in QA, then you can start with our DPR reproduction?

https://github.com/castorini/pygaggle/blob/master/docs/experiments-dpr-reader.md

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/castorini/pygaggle/issues/202#issuecomment-884431147, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHMA5SKNEWMJ4F5LSAYVYIDTY4ME3ANCNFSM5AYP6AGA .

[zdaiot]

@tanyaroosta hello. Have you solved this problem?