MQuAKE

MQuAKE copied to clipboard

Small mistake in the sample dataset record?

3

Hi there! I noticed that the sample data record in the README might contain a mistake -- just wanted to ask for clarification in case I'm misunderstanding something. The `questions`...

kmeng01

Hi, thanks for the amazing project! Up to now, this repo does not contain the evaluation scripts on GPT-J. Noticing that the MQUAKE dataset is built and filtered such that...

AOZMH

Thanks for your great work to the community! > We manually select 37 common relations and consider a subgraph that solely comprises these relations and the top 20% of common...

tbozhong

Request for source code

2

How do you train the MQuake-CF Dataset on ROME, MEMIT, please provide the source code. Then, how do you evaluate?

msumitaml

Hi! I'm particularly interested in the dataset and would appreciate it if you could consider open sourcing the code responsible for generating it. This would greatly assist me in testing...

Jometeorie

Can you release the codes for evaluation and training hyperparameters?

2

I am trying to reproduce the results on MQUAKE-T and found the multihop results for "Base" are way less (16.22/22.59 for multihop and cot) than reported in Table 4. And...

ShuoZhangXJTU

How to use the Vicuna model?

2

The Vicuna model will generate some unrelated output, so how to control the max_lrngth in : model.generate(inputs.input_ids.cuda(), max_length=??)

sev777

Hi, I would like to know how you get the original data from WikiData. Can you provide some ideas or codes?

Luoyang144