使用iTransformer遇到问题

[JKYtydt]

What happened + What you expected to happen

Traceback (most recent call last): File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 541, in _run_script exec(code, module.dict) File "/sdc/jky/llm_demo/pages/1_Training.py", line 90, in Y_hat_insample = nf.predict_insample(step_size=pred_len) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/neuralforecast/core.py", line 1175, in predict_insample fcsts[:, col_idx : (col_idx + output_length)] = model_fcsts ValueError: could not broadcast input array from shape (1404,1) into shape (1440,1) 在进行样本内预测时发生了报错，但是书写代码并没有发现问题

Versions / Dependencies

python=3.8 neuralforecast=1.7.2

Reproduction script

nf = NeuralForecast( models=[model], freq=freq )

    nf.fit(df=train_df, val_size=pred_len)

    if not os.path.exists(save_path):
        os.makedirs(save_path)

    nf.save(path=save_path,
            model_index=None,
            overwrite=True,
            save_dataset=True)

    Y_hat_insample = nf.predict_insample(step_size=pred_len)
    print('Y_hat_insample' ,Y_hat_insample.head(),Y_hat_insample.shape)
    Y_hat_insample = pd.DataFrame(Y_hat_insample)

Issue Severity

None

[elephaint]

Hi, thanks for using neuralforecast.

Can you include a fully reproducible example of code (a piece of code that I can copy-paste and run standalone on my machine), demonstrating the error? Otherwise it's very difficult for me to help.

[JKYtydt]

您好，感谢您使用神经预测。

您能否提供一个完全可重现的代码示例（一段我可以复制粘贴并在我的计算机上独立运行的代码）来演示该错误？不然我很难帮忙。

您好，这是完整的代码，麻烦你了 train_df = pd.read_csv('/sdc/jky/llm_demo/coin_BinanceCoin.csv',encoding='utf-8') print('etth1', train_df.head()) model = iTransformer(h=12, input_size=36, n_series=8, hidden_size=128, n_heads=8, e_layers=2, d_layers=1, d_ff=4, factor=1, dropout=0.1, use_norm=True, loss=MSE(), valid_loss=MSE(), early_stop_patience_steps=3, batch_size=24, max_steps=5)

nf = NeuralForecast( models=[model], freq='D' )

nf.fit(df=train_df, val_size=12)

nf.save(path=save_path, model_index=None, overwrite=True, save_dataset=True) print('模型保存完成') Y_hat_insample = nf.predict_insample(step_size=pred_len) print('Y_hat_insample' ,Y_hat_insample.head(),Y_hat_insample.shape) Y_hat_insample = pd.DataFrame(Y_hat_insample)

Y_hat_insample['unique_id'] = Y_hat_insample.index mae = mae(Y_hat_insample,models=['iTransformer'],id_col='unique_id') mse = mse(Y_hat_insample,models=['iTransformer'],id_col='unique_id') validation_df = pd.DataFrame(data={'MAE': mae['iTransformer'], 'MSE': mse['iTransformer']})

[JKYtydt]

不好意思，我又遇到一个新的问题，在进行iTransformer预测时发现torch版本不能低于2.1.0，如果低于会有如下报错 Traceback (most recent call last): File "/data/anaconda3/envs/time_py/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 600, in _run_script exec(code, module.dict) File "/sdc/jky/llm_demo/pages/2_Forecast.py", line 40, in nf2 = NeuralForecast.load(path=load_path) File "/data/anaconda3/envs/time_py/lib/python3.9/site-packages/neuralforecast/core.py", line 1357, in load loaded_model = MODEL_FILENAME_DICT[model_class_name].load( File "/data/anaconda3/envs/time_py/lib/python3.9/site-packages/neuralforecast/common/_base_model.py", line 351, in load model.load_state_dict(content["state_dict"], strict=True, assign=True) TypeError: load_state_dict() got an unexpected keyword argument 'assign' 但是如果我把版本更新到torch=2.1.0，又不能进行iTransformer的训练，也就是训练要求版本不能高于torch=2.0.1，如果高于会有如下报错 Traceback (most recent call last): File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 600, in _run_script exec(code, module.dict) File "/sdc/jky/llm_demo/pages/1_Training.py", line 80, in nf.fit(df=train_df, val_size=pred_len) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/neuralforecast/core.py", line 462, in fit self.models[i] = model.fit( File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/neuralforecast/common/_base_multivariate.py", line 537, in fit return self._fit( File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/neuralforecast/common/_base_model.py", line 219, in _fit trainer.fit(model, datamodule=datamodule) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 544, in fit call._call_and_handle_interrupt( File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt return trainer_fn(*args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 580, in _fit_impl self._run(model, ckpt_path=ckpt_path) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 987, in _run results = self._run_stage() File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1033, in _run_stage self.fit_loop.run() File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/fit_loop.py", line 205, in run self.advance() File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/fit_loop.py", line 363, in advance self.epoch_loop.run(self._data_fetcher) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/training_epoch_loop.py", line 140, in run self.advance(data_fetcher) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/training_epoch_loop.py", line 250, in advance batch_output = self.automatic_optimization.run(trainer.optimizers[0], batch_idx, kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 190, in run self._optimizer_step(batch_idx, closure) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 268, in _optimizer_step call._call_lightning_module_hook( File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 157, in _call_lightning_module_hook output = fn(*args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/core/module.py", line 1303, in optimizer_step optimizer.step(closure=optimizer_closure) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/core/optimizer.py", line 152, in step step_output = self._strategy.optimizer_step(self._optimizer, closure, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/strategies/strategy.py", line 239, in optimizer_step return self.precision_plugin.optimizer_step(optimizer, model=model, closure=closure, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/plugins/precision/precision.py", line 122, in optimizer_step return optimizer.step(closure=closure, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/torch/optim/lr_scheduler.py", line 68, in wrapper return wrapped(*args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/torch/optim/optimizer.py", line 373, in wrapper out = func(*args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/torch/optim/optimizer.py", line 76, in _use_grad ret = func(self, *args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/torch/optim/adam.py", line 143, in step loss = closure() File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/plugins/precision/precision.py", line 108, in _wrap_closure closure_result = closure() File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 144, in call self._result = self.closure(*args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 138, in closure self._backward_fn(step_output.closure_loss) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 239, in backward_fn call._call_strategy_hook(self.trainer, "backward", loss, optimizer) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 309, in _call_strategy_hook output = fn(*args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/strategies/strategy.py", line 213, in backward self.precision_plugin.backward(closure_loss, self.lightning_module, optimizer, *args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/plugins/precision/precision.py", line 72, in backward model.backward(tensor, *args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/pytorch_lightning/core/module.py", line 1090, in backward loss.backward(*args, **kwargs) File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/torch/_tensor.py", line 492, in backward torch.autograd.backward( File "/data/anaconda3/envs/time_py/lib/python3.8/site-packages/torch/autograd/init.py", line 251, in backward Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass RuntimeError: GET was unable to find an engine to execute this computation

[elephaint]

Can you upgrade your Pytorch version to the latest version? I think you might have PyTorch 2.0.0 and it seems there is an issue with our code in that version. So upgrading to 2.1+ should fix the issue.

[JKYtydt]

你能将你的 Pytorch 版本升级到最新版本吗？我认为你可能使用的是 PyTorch 2.0.0，而且该版本的代码似乎存在问题。因此升级到 2.1+ 应该可以解决问题。

1、您好，您说的我都已经试过了，如果版本更新到2.1+以上，则无法训练，报错信息我已经粘贴在上述问题中 2、关于第一个问题，我依旧无法进行样本内预测

[eye4got]

I was able to reproduce the same error and it appears to be the result of not having consistent sample sizes for each unique_id. cchallu explains potential solutions very effectively in #718. I do think some kind of check during training would be helpful.

[JKYtydt]

@eye4got 您好，非常感谢您提供的建议，您方便告诉一下具体的解决办法么

[eye4got]

@JKYtydt you need to make every unique_id have the same number of time steps. So you can either: 1) cut all the timeseries down to the same length and/or not include some shorter timeseries, so that they are all say 100 days long; 2) use some kind of interpolation or imputation method to fill missing data so they are all the same length. I honestly switched to using the darts package when I found out that neuralforecast doesn't support multivariate forecasting for TFT models. It is much more flexible when it comes to restrictions like this

[JKYtydt]

@eye4got 您好，我的这个数据集unique_id是唯一的，也没有缺失值，但是它仍然存在这个报错，且发现这个形状的差别刚好是输入的序列长度，这有什么关联么

test.py 68 <module>
Y_hat_insample = nf.predict_insample(step_size=12)

core.py 1175 predict_insample
fcsts[:, col_idx : (col_idx + output_length)] = model_fcsts

ValueError:
could not broadcast input array from shape (1416,1) into shape (1440,1)