SapienzaNLP
Whether can you provide PROCESSED NYT dataset used for entity relation extraction training
I processed the data according to the tutorial and reproduced the training process. But, the f1 score is only 83, not the 95 in paper, and pretrained RE model 's performance is not like the value in paper . So could you can provide the correctly processed data of NYT and the correct evaluation code of EL? Looking forward to your reply.
Hi there, sorry for the late reply. Which processed data do you mean? For training you need to follow the readme and create it for both the retriever and the reader. I can probably provide either of them, but it's just running those scripts. For evaluation it is simpler, you can provide the sentences and use the already trained models in HF (https://huggingface.co/sapienzanlp/relik-relation-extraction-nyt-large) to obtain the values in the paper.
Be aware that by training both components from your side you should see some fluctuation (but not 10 points). Can you provide more information on what you tried? (i.e. trained both retriever and reader, just the reader?, which config for training)
