Replicating Result on WebNLG

[vvhj]

Thanks for your nice work.

I am try to replicate result on webNLG, but the finnal epochs of checkpoint is only 11270, different from 20000. This results in a significant difference in the accuracy of the reproduction compared to your results.

Here is the my instruct:

python -m torch.distributed.launch --nproc_per_node=1 src/gpt2_ft.py
--train_data ./data/webnlg_challenge_2017/train.jsonl
--valid_data ./data/webnlg_challenge_2017/valid.jsonl
--train_batch_size 8
--grad_acc 1
--valid_batch_size 4
--seq_len 512
--model_card gpt2.md
--init_checkpoint ./pretrained_checkpoints/gpt2-medium-pytorch_model.bin
--platform local
--clip 0.0
--lr 0.0002
--weight_decay 0.01
--correct_bias
--adam_beta2 0.999
--scheduler linear
--warmup_step 500
--max_epoch 5
--save_interval 1000
--lora_dim 4
--lora_alpha 32
--lora_dropout 0.1
--label_smooth 0.1
--work_dir ./trained_models/GPT2_M/webnlgv9
--random_seed 110

[edwardjhu]

Are you saying that the checkpoint we uploaded is from iteration 11270, not 20000? I need to double check but it's possible we picked the best performing checkpoint, which is not necessarily the final one, following prior work.

[RayCyder]

yes , same problem;