Can't reproduce the result of PEMS03_96_96 task

[Secilia-Cxy]

Hi dear authors, thank you for your contribution in this article. However, now I have trouble in reproducing the result of PEMS03_96_96 task using the script provided in scripts/multivariate_forecasting/PEMS/iTransformer_03.sh. I didn't change any code after downloading the repo and used the data you provide. Here are my logs: Args in experiment: Namespace(activation='gelu', batch_size=32, c_out=358, channel_independence=False, checkpoints='./checkpoints/', class_strategy='projection', d_ff=512, d_layers=1, d_model=512, data='PEMS', data_path='PEMS03.npz', dec_in=358, des='Exp', devices='0,1,2,3', distil=True, do_predict=False, dropout=0.1, e_layers=4, efficient_training=False, embed='timeF', enc_in=358, exp_name='MTSF', factor=1, features='M', freq='h', gpu=0, inverse=False, is_training=1, itr=1, label_len=48, learning_rate=0.001, loss='MSE', lradj='type1', model='iTransformer', model_id='PEMS03_96_96', moving_avg=25, n_heads=8, num_workers=10, output_attention=False, partial_start_index=0, patience=3, pred_len=96, root_path='./dataset/PEMS/', seq_len=96, target='OT', target_data_path='electricity.csv', target_root_path='./data/electricity/', train_epochs=10, use_amp=False, use_gpu=True, use_multi_gpu=False, use_norm=True) Use GPU: cuda:0

start training : PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0>>>>>>>>>>>>>>>>>>>>>>>>>> train 15533 val 5051 test 5051 iters: 100, epoch: 1 | loss: 0.6254231 speed: 0.0592s/iter; left time: 281.3028s iters: 200, epoch: 1 | loss: 0.5153803 speed: 0.0428s/iter; left time: 198.9109s iters: 300, epoch: 1 | loss: 0.5594893 speed: 0.0432s/iter; left time: 196.7037s iters: 400, epoch: 1 | loss: 0.5180222 speed: 0.0436s/iter; left time: 193.9329s Epoch: 1 cost time: 22.676926612854004 Epoch: 1, Steps: 485 | Train Loss: 0.6499067 Vali Loss: 2.3864691 Test Loss: 2.3295264 Validation loss decreased (inf --> 2.386469). Saving model ... Updating learning rate to 0.001 iters: 100, epoch: 2 | loss: 2.3072736 speed: 0.5208s/iter; left time: 2221.7887s iters: 200, epoch: 2 | loss: 1.6124172 speed: 0.0418s/iter; left time: 174.2366s iters: 300, epoch: 2 | loss: 1.4063895 speed: 0.0422s/iter; left time: 171.6863s iters: 400, epoch: 2 | loss: 1.6972579 speed: 0.0424s/iter; left time: 168.3158s Epoch: 2 cost time: 21.366225242614746 Epoch: 2, Steps: 485 | Train Loss: 1.7878543 Vali Loss: 2.2568407 Test Loss: 2.1668501 Validation loss decreased (2.386469 --> 2.256841). Saving model ... Updating learning rate to 0.0005 iters: 100, epoch: 3 | loss: 1.9239104 speed: 0.5242s/iter; left time: 1981.9011s iters: 200, epoch: 3 | loss: 1.8586614 speed: 0.0418s/iter; left time: 153.7583s iters: 300, epoch: 3 | loss: 2.1840055 speed: 0.0424s/iter; left time: 151.8389s iters: 400, epoch: 3 | loss: 2.3375337 speed: 0.0426s/iter; left time: 148.3466s Epoch: 3 cost time: 21.4084894657135 Epoch: 3, Steps: 485 | Train Loss: 1.9079526 Vali Loss: 2.0450311 Test Loss: 1.9737376 Validation loss decreased (2.256841 --> 2.045031). Saving model ... Updating learning rate to 0.00025 iters: 100, epoch: 4 | loss: 1.9168335 speed: 0.5273s/iter; left time: 1737.9915s iters: 200, epoch: 4 | loss: 1.7213866 speed: 0.0422s/iter; left time: 134.7857s iters: 300, epoch: 4 | loss: 2.4566705 speed: 0.0425s/iter; left time: 131.6806s iters: 400, epoch: 4 | loss: 1.9949968 speed: 0.0428s/iter; left time: 128.2878s Epoch: 4 cost time: 21.496201038360596 Epoch: 4, Steps: 485 | Train Loss: 2.0230194 Vali Loss: 2.0335276 Test Loss: 1.9665655 Validation loss decreased (2.045031 --> 2.033528). Saving model ... Updating learning rate to 0.000125 iters: 100, epoch: 5 | loss: 1.7979079 speed: 0.5207s/iter; left time: 1463.5948s iters: 200, epoch: 5 | loss: 2.1647246 speed: 0.0422s/iter; left time: 114.4971s iters: 300, epoch: 5 | loss: 2.3066046 speed: 0.0425s/iter; left time: 110.9998s iters: 400, epoch: 5 | loss: 1.9180266 speed: 0.0428s/iter; left time: 107.3851s Epoch: 5 cost time: 21.60309934616089 Epoch: 5, Steps: 485 | Train Loss: 2.0156756 Vali Loss: 2.0310166 Test Loss: 1.9661391 Validation loss decreased (2.033528 --> 2.031017). Saving model ... Updating learning rate to 6.25e-05 iters: 100, epoch: 6 | loss: 1.8521897 speed: 0.5214s/iter; left time: 1212.6685s iters: 200, epoch: 6 | loss: 1.7516365 speed: 0.0421s/iter; left time: 93.6896s iters: 300, epoch: 6 | loss: 1.8441006 speed: 0.0424s/iter; left time: 90.1091s iters: 400, epoch: 6 | loss: 1.7079446 speed: 0.0426s/iter; left time: 86.2513s Epoch: 6 cost time: 21.40552520751953 Epoch: 6, Steps: 485 | Train Loss: 1.8906130 Vali Loss: 2.3260319 Test Loss: 2.2649562 EarlyStopping counter: 1 out of 3 Updating learning rate to 3.125e-05 iters: 100, epoch: 7 | loss: 1.6522958 speed: 0.5173s/iter; left time: 952.3502s iters: 200, epoch: 7 | loss: 1.4734029 speed: 0.0420s/iter; left time: 73.1873s iters: 300, epoch: 7 | loss: 1.4864014 speed: 0.0424s/iter; left time: 69.5053s iters: 400, epoch: 7 | loss: 1.1650115 speed: 0.0426s/iter; left time: 65.6658s Epoch: 7 cost time: 21.409071683883667 Epoch: 7, Steps: 485 | Train Loss: 1.4858343 Vali Loss: 2.6214731 Test Loss: 2.5515866 EarlyStopping counter: 2 out of 3 Updating learning rate to 1.5625e-05 iters: 100, epoch: 8 | loss: 1.3040706 speed: 0.5174s/iter; left time: 701.6062s iters: 200, epoch: 8 | loss: 1.2672142 speed: 0.0420s/iter; left time: 52.7530s iters: 300, epoch: 8 | loss: 1.3993770 speed: 0.0423s/iter; left time: 48.9095s iters: 400, epoch: 8 | loss: 1.7092217 speed: 0.0426s/iter; left time: 44.9640s Epoch: 8 cost time: 21.37936282157898 Epoch: 8, Steps: 485 | Train Loss: 1.4200343 Vali Loss: 2.6516719 Test Loss: 2.5801837 EarlyStopping counter: 3 out of 3 Early stopping testing : PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< test 5051 test shape: (5051, 1, 96, 358) (5051, 1, 96, 358) test shape: (5051, 96, 358) (5051, 96, 358) mse:1.9661425352096558, mae:1.2062638998031616