Can't get the same result on TREC dataset

[BD-MF]

According to the parameter configuration in the paper, I cannot get the results similar to those in the paper (the effect is very poor in the TREC data set, only 51%). Is there any problem with the parameter configuration in the sh file I run?

task=trec gpu=2 method=selftrain max_seq_len=64 batch_size=16 echo ${method} python3 main.py
--do_train
--do_eval
--task=${task}
--train_file=train_data.json
--dev_file=dev_data.json
--test_file=test_data.json
--unlabel_file=unlabeled_data.json
--task_type=tc
--data_dir="data/${task}"
--rule=1
--logging_steps=100
--self_train_logging_steps=100
--gpu="${gpu}"
--num_train_epochs=3
--weight_decay=1e-4
--method=${method}
--batch_size=${batch_size}
--max_seq_len=${max_seq_len}
--auto_load=1
--self_training_power=2
--self_training_contrastive_weight=1
--distmetric='cos'
--max_steps=500
--self_training_max_step=2500
--self_training_update_period=30
--self_training_eps=0.3
--self_training_confreg=0.05 \

[yueyu1030]

Hi, here I recommend you to first tune the T1 to get a good initialization (which should be around 65% as reported in the paper), then with a good initialization, you also need to tune the self_training_eps (maybe from [0.3, 0.6]) and self_training_update_period (from [30, 60, 100], etc.) to improve the result of self-training. Also, our results are obtained using the released script with 2080Ti GPUs. Different hardware may also result in different performances.

Here is a screenshot of my reproduction results several months ago.