neuralforecast
neuralforecast copied to clipboard
Published
20 hours ago •
Nixtla
Nixtla
TimesNet: Runtime error when trying to forecast more than 12 months
What happened + What you expected to happen
I am using the example provided in the documentation for the Times Net model that uses the Air passengers monthly data. While experimenting, i tried to increase the forecast horizon (h) above 12 months but i keep getting a Runtime error that the maximum size of tensor at dimension 2 is not enough. The model is working fine if i use h<= 12.
In the attached script i have used h=24 and input_size=48 and the error i get is the following:
RuntimeError Traceback (most recent call last)
Cell In[187], [line 20](vscode-notebook-cell:?execution_count=187&line=20)
[4](vscode-notebook-cell:?execution_count=187&line=4) model = TimesNet(h=24,
[5](vscode-notebook-cell:?execution_count=187&line=5) input_size=48,
[6](vscode-notebook-cell:?execution_count=187&line=6) hidden_size = 16,
(...)
[13](vscode-notebook-cell:?execution_count=187&line=13) val_check_steps=50,
[14](vscode-notebook-cell:?execution_count=187&line=14) early_stop_patience_steps=2)
[16](vscode-notebook-cell:?execution_count=187&line=16) nf = NeuralForecast(
[17](vscode-notebook-cell:?execution_count=187&line=17) models=[model],
[18](vscode-notebook-cell:?execution_count=187&line=18) freq='M'
[19](vscode-notebook-cell:?execution_count=187&line=19) )
---> [20](vscode-notebook-cell:?execution_count=187&line=20) nf.fit(df=Y_train_df, static_df=AirPassengersStatic, val_size=12)
[21](vscode-notebook-cell:?execution_count=187&line=21) forecasts = nf.predict(futr_df=Y_test_df)
File [c:\Users\.venv\lib\site-packages\neuralforecast\core.py:486, in NeuralForecast.fit(self, df, static_df, val_size, sort_df, use_init_models, verbose, id_col, time_col, target_col, distributed_config)
[483](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:483) self._reset_models()
[485](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:485) for i, model in enumerate(self.models):
--> [486](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:486) self.models[i] = model.fit(
[487](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:487) self.dataset, val_size=val_size, distributed_config=distributed_config
[488](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:488) )
[490](file:///C:/Users/.venv/lib/site-packages/neuralforecast/core.py:490) self._fitted = True
File [c:\Users\.venv\lib\site-packages\neuralforecast\common\_base_windows.py:661, in BaseWindows.fit(self, dataset, val_size, test_size, random_seed, distributed_config)
[632](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:632) def fit(
[633](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:633) self,
[634](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:634) dataset,
(...)
[638](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:638) distributed_config=None,
[639](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:639) ):
[640](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:640) """Fit.
[641](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:641)
[642](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:642) The `fit` method, optimizes the neural network's weights using the
(...)
[659](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:659) `test_size`: int, test size for temporal cross-validation.<br>
[660](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:660) """
--> [661](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:661) return self._fit(
[662](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:662) dataset=dataset,
[663](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:663) batch_size=self.batch_size,
[664](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:664) valid_batch_size=self.valid_batch_size,
[665](file:///C:/Users.venv/lib/site-packages/neuralforecast/common/_base_windows.py:665) val_size=val_size,
[666](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:666) test_size=test_size,
[667](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:667) random_seed=random_seed,
[668](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:668) distributed_config=distributed_config,
[669](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:669) )
File [c:\Users\.venv\lib\site-packages\neuralforecast\common\_base_model.py:357, in BaseModel._fit(self, dataset, batch_size, valid_batch_size, val_size, test_size, random_seed, shuffle_train, distributed_config)
[355](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:355) model = self
[356](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:356) trainer = pl.Trainer(**model.trainer_kwargs)
--> [357](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:357) trainer.fit(model, datamodule=datamodule)
[358](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:358) model.metrics = trainer.callback_metrics
[359](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_model.py:359) model.__dict__.pop("_trainer", None)
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:543, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
[541](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:541) self.state.status = TrainerStatus.RUNNING
[542](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:542) self.training = True
--> [543](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:543) call._call_and_handle_interrupt(
[544](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:544) self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
[545](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:545) )
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\call.py:44, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
[42](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:42) if trainer.strategy.launcher is not None:
[43](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:43) return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
---> [44](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:44) return trainer_fn(*args, **kwargs)
[46](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:46) except _TunerExitException:
[47](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:47) _call_teardown_hook(trainer)
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:579, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
[572](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:572) assert self.state.fn is not None
[573](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:573) ckpt_path = self._checkpoint_connector._select_ckpt_path(
[574](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:574) self.state.fn,
[575](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:575) ckpt_path,
[576](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:576) model_provided=True,
[577](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:577) model_connected=self.lightning_module is not None,
[578](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:578) )
--> [579](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:579) self._run(model, ckpt_path=ckpt_path)
[581](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:581) assert self.state.stopped
[582](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:582) self.training = False
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:986, in Trainer._run(self, model, ckpt_path)
[981](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:981) self._signal_connector.register_signal_handlers()
[983](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:983) # ----------------------------
[984](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:984) # RUN THE TRAINER
[985](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:985) # ----------------------------
--> [986](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:986) results = self._run_stage()
[988](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:988) # ----------------------------
[989](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:989) # POST-Training CLEAN UP
[990](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:990) # ----------------------------
[991](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:991) log.debug(f"{self.__class__.__name__}: trainer tearing down")
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:1028, in Trainer._run_stage(self)
[1026](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1026) if self.training:
[1027](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1027) with isolate_rng():
-> [1028](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1028) self._run_sanity_check()
[1029](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1029) with torch.autograd.set_detect_anomaly(self._detect_anomaly):
[1030](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1030) self.fit_loop.run()
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:1057, in Trainer._run_sanity_check(self)
[1054](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1054) call._call_callback_hooks(self, "on_sanity_check_start")
[1056](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1056) # run eval step
-> [1057](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1057) val_loop.run()
[1059](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1059) call._call_callback_hooks(self, "on_sanity_check_end")
[1061](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/trainer.py:1061) # reset logger connector
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\loops\utilities.py:182, in _no_grad_context.<locals>._decorator(self, *args, **kwargs)
[180](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/utilities.py:180) context_manager = torch.no_grad
[181](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/utilities.py:181) with context_manager():
--> [182](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/utilities.py:182) return loop_run(self, *args, **kwargs)
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\loops\evaluation_loop.py:135, in _EvaluationLoop.run(self)
[133](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:133) self.batch_progress.is_last_batch = data_fetcher.done
[134](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:134) # run step hooks
--> [135](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:135) self._evaluation_step(batch, batch_idx, dataloader_idx, dataloader_iter)
[136](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:136) except StopIteration:
[137](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:137) # this needs to wrap the `*_step` call too (not just `next`) for `dataloader_iter` support
[138](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:138) break
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\loops\evaluation_loop.py:396, in _EvaluationLoop._evaluation_step(self, batch, batch_idx, dataloader_idx, dataloader_iter)
[390](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:390) hook_name = "test_step" if trainer.testing else "validation_step"
[391](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:391) step_args = (
[392](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:392) self._build_step_args_from_hook_kwargs(hook_kwargs, hook_name)
[393](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:393) if not using_dataloader_iter
[394](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:394) else (dataloader_iter,)
[395](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:395) )
--> [396](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:396) output = call._call_strategy_hook(trainer, hook_name, *step_args)
[398](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:398) self.batch_progress.increment_processed()
[400](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:400) if using_dataloader_iter:
[401](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/loops/evaluation_loop.py:401) # update the hook kwargs now that the step method might have consumed the iterator
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\trainer\call.py:311, in _call_strategy_hook(trainer, hook_name, *args, **kwargs)
[308](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:308) return None
[310](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:310) with trainer.profiler.profile(f"[Strategy]{trainer.strategy.__class__.__name__}.{hook_name}"):
--> [311](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:311) output = fn(*args, **kwargs)
[313](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:313) # restore current_fx when nested context
[314](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/trainer/call.py:314) pl_module._current_fx_name = prev_fx_name
File [c:\Users\.venv\lib\site-packages\pytorch_lightning\strategies\strategy.py:411, in Strategy.validation_step(self, *args, **kwargs)
[409](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/strategies/strategy.py:409) if self.model != self.lightning_module:
[410](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/strategies/strategy.py:410) return self._forward_redirection(self.model, self.lightning_module, "validation_step", *args, **kwargs)
--> [411](file:///C:/Users/.venv/lib/site-packages/pytorch_lightning/strategies/strategy.py:411) return self.lightning_module.validation_step(*args, **kwargs)
File [c:\Users\.venv\lib\site-packages\neuralforecast\common\_base_windows.py:492, in BaseWindows.validation_step(self, batch, batch_idx)
[489](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:489) return np.nan
[491](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:491) # TODO: Hack to compute number of windows
--> [492](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:492) windows = self._create_windows(batch, step="val")
[493](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:493) n_windows = len(windows["temporal"])
[494](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:494) y_idx = batch["y_idx"]
File [c:\Users\.venv\lib\site-packages\neuralforecast\common\_base_windows.py:248, in BaseWindows._create_windows(self, batch, step, w_idxs)
[245](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:245) padder_right = nn.ConstantPad1d(padding=(0, self.h), value=0)
[246](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:246) temporal = padder_right(temporal)
--> [248](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:248) windows = temporal.unfold(
[249](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:249) dimension=-1, size=window_size, step=predict_step_size
[250](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:250) )
[252](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:252) # [batch, channels, windows, window_size] 0, 1, 2, 3
[253](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:253) # -> [batch * windows, window_size, channels] 0, 2, 3, 1
[254](file:///C:/Users/.venv/lib/site-packages/neuralforecast/common/_base_windows.py:254) windows_per_serie = windows.shape[2]
RuntimeError: maximum size for tensor at dimension 2 is 60 but size is 72
Versions / Dependencies
pandas==1.5.3 torch==2.0.0 neuralforecast==1.7.3 numpy==1.23.5 python== 3.8.10
Reproduction script
import numpy as np
import pandas as pd
from neuralforecast import NeuralForecast
from neuralforecast.losses.pytorch import DistributionLoss
from neuralforecast.utils import AirPassengers, AirPassengersPanel, AirPassengersStatic, augment_calendar_df
from neuralforecast.models import TimesNet
#### Augment the DataFrame with calendar features
AirPassengersPanel, calendar_cols = augment_calendar_df(df=AirPassengersPanel, freq='M')
#### Split the data into training and testing sets
Y_train_df = AirPassengersPanel[AirPassengersPanel.ds < AirPassengersPanel['ds'].values[-24]]
Y_test_df = AirPassengersPanel[AirPassengersPanel.ds >= AirPassengersPanel['ds'].values[-24]].reset_index(drop=True)
#### TimesNet model
model = TimesNet(h=24,
input_size=48,
hidden_size = 16,
conv_hidden_size = 32,
loss=DistributionLoss(distribution='Normal', level=[80, 90]),
futr_exog_list=calendar_cols,
scaler_type='standard',
learning_rate=1e-3,
max_steps=5,
val_check_steps=50,
early_stop_patience_steps=2)
nf = NeuralForecast(
models=[model],
freq='M'
)
nf.fit(df=Y_train_df, static_df=AirPassengersStatic, val_size=12)
forecasts = nf.predict(futr_df=Y_test_df)
Issue Severity
High: It blocks me from completing my task.
