How to reproduce results of the paper？

[Devil-Ideal]

I read the paper and downloaded the AG news dataset，and tested PET model on it，but there is a great margin between my results and the author's results. I set parameters as below. To be specifically, I just used 10 examples for train(10 shot),model type is roberta,model_name_or_path is roberta-large,and I used all patterns for AG news. I did not change other parameters. And here is my results: ==============my results=============== acc-p0: 0.6450877192982456 +- 0.053859095516898825 acc-p1: 0.7874561403508772 +- 0.01841603941808791 acc-p2: 0.5642543859649123 +- 0.06912621607498706 acc-p3: 0.6119298245614034 +- 0.09528808314997761 acc-p4: 0.7537719298245614 +- 0.07473549078343446 acc-all-p: 0.6725 +- 0.10462149351553651 ===============parameters setting=========== parser.add_argument("--train_examples", default=10, type=int, help="The total number of train examples to use, where -1 equals all examples.") parser.add_argument("--method", required=False, default='pet', choices=['pet', 'ipet', 'sequence_classifier'], help="The training method to use. Either regular sequence classification, PET or iPET.") parser.add_argument("--data_dir", default="./agnews/", type=str, required=False, help="The input data dir. Should contain the data files for the task.") parser.add_argument("--model_type", default="roberta", type=str, required=False, choices=MODEL_CLASSES.keys(), help="The type of the pretrained language model to use") parser.add_argument("--model_name_or_path", default="roberta-large", type=str, required=False, help="Path to the pre-trained model or shortcut name") parser.add_argument("--task_name", default="agnews", type=str, required=False, choices=PROCESSORS.keys(), help="The name of the task to train/evaluate on") parser.add_argument("--output_dir", default="./output/", type=str, required=False, help="The output directory where the model predictions and checkpoints will be written")

# PET-specific optional parameters
parser.add_argument("--wrapper_type", default="mlm", choices=WRAPPER_TYPES,
                    help="The wrapper type. Set this to 'mlm' for a masked language model like BERT or to 'plm' "
                         "for a permuted language model like XLNet (only for PET)")
parser.add_argument("--pattern_ids", default=[0,1,2,3,4], type=int, nargs='+',
                    help="The ids of the PVPs to be used (only for PET)")