Akari Asai

Sorry I have overlooked this issue! As mentioned in the paper Section 2.1.4, the answer translations have been conducted for the evaluation set only so that the `xor_train_full.json` includes the...

Hi, sorry for my late response! Could you share the command you are running and in which dataset you have that issue? I think I have seen the same issue...

Hi, so sorry for my late response! Could you share the entire error message so that I could see which line is the error happening?

I'm closing this issue now but please feel free to re-open if you have any followup questions!

Do you mind providing the vllm version? Not directly about Self-RAG, but I've recently encountered similar issues when I was loading Mixtral models (e.g., examples are all blank) and I...

Hi, thank you so much for reporting! Hmm, the citation rec and precision particularly look low... Let me check in on this tomorrow.

Sorry for my late response! I was busy with other commitments in the past two weeks. I think the issue might have happened due to some code changes I did...

Sorry for my late response! This is the link to our 7B prediction results: [Google Drive](https://drive.google.com/file/d/190YHd-7fCzZ7ml31WuCKwaFpHqBl6UC0/view?usp=drive_link) Here's the output of the asqa eval.py script. ``` { "length": 29.829113924050635, "str_em": 29.957805907172997,...

Sorry for being late on this issue as I was being busy with helping to wrap up some other projects and traveling in the past weeks. I can upload the...

Here's the 13B predictions ([google drive](https://drive.google.com/file/d/1p-QKCe8v1akoCNraoEaTUHcaMRV3DBxr/view?usp=drive_link)) and results: ``` {'length': 27.029535864978904, 'str_em': 31.66139240506329, 'str_hit': 8.438818565400844, 'rougeLsum': 36.0146483715914, 'QA-EM': 20.386779184247537, 'QA-F1': 26.404630941269915, 'QA-Hit': 2.9535864978902953, 'mauve': 71.59056482735427, 'citation_rec': 70.35387783805504, 'citation_prec': 71.26280892103678} ```