LTSF-Linear icon indicating copy to clipboard operation
LTSF-Linear copied to clipboard

about the final results in the paper [2]

Open super-shayan opened this issue 2 years ago • 3 comments

Hello,

I am having trouble reproducing some results for the DLinear model, could the authors please comment if something is wrong with the results for the datasets ETTh2

datasets in the paper reproduced
0.399 0.397
ETTh1 0.416 0.418
0.443 0.436
0.49 0.495
0.353 0.401
ETTh2 0.418 0.463
0.465 0.502
0.551 0.586
0.343 0.345
ETTm1 0.365 0.364
0.386 0.389
0.421 0.420
0.26 0.261
ETTm2 0.303 0.316
0.342 0.366
0.421 0.436
0.237 0.239
Electricity 0.249 0.252
0.267 0.273
0.301 0.300
0.203 0.204
Exchange 0.293 0.312
0.414 0.417
0.601 0.808
0.282 0.291
Traffic 0.287 0.291
0.296 0.299
0.315 0.317
0.237 0.234
Weather 0.282 0.276
0.319 0.310
0.362 0.371

I used the hyperparameters given in the code repository for the sequence length 336 for the ETTh2 task, given here: https://github.com/cure-lab/DLinear/blob/main/scripts/EXP-LongForecasting/DLinear/etth2.sh

super-shayan avatar Aug 02 '22 17:08 super-shayan

Hi @super-shayan, Thanks for your kind feedback! May I know the version of your experimental environments (e.g., Pytorch, CUDA, Cudnn)? If you use the same version with us under a fixed random seed, the results should be the same as ours.

ailingzengzzz avatar Aug 04 '22 16:08 ailingzengzzz

Hello, Thank you for your response... but that is my point, I might be using different versions of the libraries you listed, but the numbers are still expected to be in the same ranges, and for most datasets that is the case indeed, but only for ETTh2 I am seeing a more of a difference, that is why I suspect the hyperparameters for ETTh2 might not be the same as listed in the GitHub repository file: https://github.com/cure-lab/DLinear/blob/main/scripts/EXP-LongForecasting/DLinear/etth2.sh

super-shayan avatar Aug 06 '22 12:08 super-shayan

image Hi, the parameter is correct. You can also try lr=0.005, which can achieve similar results.

mixiancmx avatar Aug 06 '22 15:08 mixiancmx