issue reproducing the results

[IssamLaradji]

Hi there, I have ran the following command from run.sh, in an attempt to reproduce the results for roberta but I am facing discrepencie.

python main.py --learning-rate 1e-5 --weight-decay 0 --batch-size 3 --epochs 21 --log-interval 400 \
    --grad-accum-steps 4 --model-type roberta --prefix 0524 --filename final --task ast

I am getting these results, which seem different from Roberta in Table 4 where Value is 73.1% instead o 53.9.

• Could you explain this discrepency And quick question?
• is "intent" in Table 4 also called Action_Accuracy in the code? Thanks!

[gsoykan]

Hello, I'm encountering a similar issue. @IssamLaradji, for your AST case, I recommend referring to Table 3, where the ACC value is listed as 67.2. However, it appears to differ significantly from your result. In my situation, I'm attempting to replicate BERT, achieving an ACC of 37.4 for value, while the ACC for action is 88.6.