Questions about the performance of the model

[paulpig]

Hello, I'm very interested in your work. However, when I train your model by myself, the performance is lower than the results reported in the paper and the results in your "log files".

The parameters are set to the default value you provide.

The experimental results I got are as follows: Twitter 2015: Twitter 2017:

I would like to ask how I can achieve the results you reported.

[jefferyYu]

Hi there,

It is natural that when you run the same code with the same parameter setting on different servers, the results are slightly different, especifically on relatively small datasets.

Actually, the parameter setting provided in the .sh file are tuned on the development set on our current server (with a NVIDIA TITAN RTX GPU), which can achieve the best performance on the dev set based on grid search. To achieve the best performance on your server, you need to tune some parameters such as train_batch_size and num_train_epochs. Besides, you may try different random seeds, i.e., seed, which will also affect the model performance on your server.

Hope it clarifies your concern.

Best, Jianfei