STSGCN icon indicating copy to clipboard operation
STSGCN copied to clipboard

about result

Open ICDI0906 opened this issue 5 years ago • 14 comments

come again~ I have run your data on my model, but the result is unbelievable. I almost get the same result with the model graph wavenet on public dataset metr-la. But on your dataset PEMS08, I get MAE: 1.57, MAPE: 3.44%, RMSE: 3.73. Your results are MAE 17.13, MAPE 10.96%, RMSE 26.80. which is much smaller. I have checked my code, but I didn't find any problem. Do you know why? can you tell me your experiment details?

ICDI0906 avatar Jan 16 '20 02:01 ICDI0906

I don't know why your results looks so wired. As far as I know, graph wavenet does not fit all dataset nicely. I run the experiment in July last year, you should check whether they have updated code, so that the result may be different. Besides, you should also check all datasets not only the PeMS08, and all experiment should be finished ten times, compute the mean and std. I dont think there is any problem in my experiment. In addition, you should compute the mean value of the PEMS08, if the mean value is around 100, do you think the 1.57 is reasonable?

Davidham3 avatar Jan 16 '20 03:01 Davidham3

I also think my result is wired.I will check again.Did you just predict speed on the PEMS08?

| | 18217060267 邮箱:[email protected] |

Signature is customized by Netease Mail Master

On 01/16/2020 11:00, Davidham3 wrote:

I don't know why your results looks so wired. As far as I know, graph wavenet does not fit all dataset nicely. I run the experiment in July last year, you should check whether they have updated code, so that the result may be different. Besides, you should also check all datasets not only the PeMS08, and all experiment should be finished ten times, compute the mean and std. I dont think there is any problem in my experiment. In addition, you should compute the mean value of the PEMS08, if the mean value is around 100, do you think the 1.57 is reasonable?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

ICDI0906 avatar Jan 16 '20 03:01 ICDI0906

Hello, I run your data using DCRNN model, the MAE is also small. In the experiment, I just change the dataset. Maybe I lose some important steps, I want to know how you do expriment with DCRNN. thanks~

ICDI0906 avatar Jan 16 '20 06:01 ICDI0906

我也感觉论文的结果有点问题,MAPE接近20%对于速度预测来说有点夸张了,会不会是计算的时候哪里出了问题?也许可以用公开的数据集METR-LA试一下

CYBruce avatar Aug 17 '20 10:08 CYBruce

Hello, I run your data using DCRNN model, the MAE is also small. In the experiment, I just change the dataset. Maybe I lose some important steps, I want to know how you do expriment with DCRNN. thanks~

我在许多公开实验集的实验过程中graph wavenet优于DCRNN优于STGCN,但这个实验结果相反

CYBruce avatar Aug 17 '20 11:08 CYBruce

是的,可能作者没有公开更多的实验细节吧

ICDI0906 avatar Aug 17 '20 11:08 ICDI0906

?也许可以用公开的数据集METR-LA试一下

请问下这篇文章不是进行的流量预测吗

estimate123 avatar Oct 01 '20 14:10 estimate123

?也许可以用公开的数据集METR-LA试一下

请问下这篇文章不是进行的流量预测吗

虽然我还没复现这篇文章,但是Graph WaveNet的性能应该是明显好于STGCN(原始)的,不管是在速度还是流量,因为原始的STGCN只有两个block,6层,而Graph WaveNet 可有8个block,至少14层以上,这参数量就不是一个级别的。都是Gated-CNN加GCN。 ——。—— lol

guokan987 avatar Nov 14 '20 11:11 guokan987

?也许可以用公开的数据集METR-LA试一下

请问下这篇文章不是进行的流量预测吗

虽然我还没复现这篇文章,但是Graph WaveNet的性能应该是明显好于STGCN(原始)的,不管是在速度还是流量,因为原始的STGCN只有两个block,6层,而Graph WaveNet 可有8个block,至少14层以上,这参数量就不是一个级别的。都是Gated-CNN加GCN。 ——。—— lol

请问你复现过ASTGCN那篇文章吗,能复现出论文里的效果吗?我训练那个模型总是过拟合,达不到ASTGCN论文里的效果

estimate123 avatar Nov 23 '20 12:11 estimate123

?也许可以用公开的数据集METR-LA试一下

请问下这篇文章不是进行的流量预测吗

虽然我还没复现这篇文章,但是Graph WaveNet的性能应该是明显好于STGCN(原始)的,不管是在速度还是流量,因为原始的STGCN只有两个block,6层,而Graph WaveNet 可有8个block,至少14层以上,这参数量就不是一个级别的。都是Gated-CNN加GCN。 ——。—— lol

请问你复现过ASTGCN那篇文章吗,能复现出论文里的效果吗?我训练那个模型总是过拟合,达不到ASTGCN论文里的效果

我记得好像训练在10-20epochs时,就达到最好的validation的结果。。。。确实有点过拟合。。。。。。效果也差点意思。。。。lol

guokan987 avatar Nov 24 '20 02:11 guokan987

好像训练在10-20epochs时,就达到最好的validation的结果。。。。确实有点过拟合。。。。。。效果也差点意思。。。。lol

是的,可能是ASTGCN模型参数太多了吧

estimate123 avatar Nov 24 '20 02:11 estimate123

据我所知,pems数据集中不仅含有flow data还有speed data等,如果误用speed数据的话,mae等评价指标有可能变得很小

xingzhinie avatar May 20 '22 12:05 xingzhinie

Graph WaveNet展示的结果是不是有问题,我这边跑的结果比论文中的结果小很多

JianSoL avatar Jun 02 '22 14:06 JianSoL

Graph WaveNet展示的结果是不是有问题,我这边跑的结果比论文中的结果小很多

lol,摊手

guokan987 avatar Jun 03 '22 02:06 guokan987