pytorch-forecasting
pytorch-forecasting copied to clipboard
Published
20 hours ago •
jdb78
jdb78
Time series forecasting with PyTorch
Results
401
pytorch-forecasting issues
Sort by
recently updated
recently updated
newest added
TimeSeriesDataSet with multiple targets
[{"_id":"632917f0a6ecce368d5d2943","body":"I have a similar issue,\r\nI'm trying to train multiple target model but do not know How to set target normalizer and group_ids\r\nThx","issue_id":1660140579668,"origin_id":1007150262,"user_origin_id":16810004,"create_time":1641533056,"update_time":1641533056,"id":1663637488839,"updated_at":"2022-09-20T01:31:28.839000Z","created_at":"2022-09-20T01:31:28.839000Z"},{"_id":"632917f0a6ecce368d5d2944","body":"@jhbale11 \r\nI made a new column as group_id and initialized it with 0. Because i've only one group.\r\nNormalizer i have not configured yet.\r\n\r\nI made other changes and they seem to be working\r\nIn the `TemporalFusionTransformer.from_dataset` part it seems that theres a list needed as output_size.\r\n```\r\noutput_size=[7, 7, 7, 7], # 4 target variables\r\n```\r\nand for loss calculations i put in a `MultiLoss`\r\n```\r\nloss=MultiLoss([QuantileLoss(), QuantileLoss(), QuantileLoss(), QuantileLoss()]),\r\n```","issue_id":1660140579668,"origin_id":1007768209,"user_origin_id":37734030,"create_time":1641592592,"update_time":1641592592,"id":1663637488844,"updated_at":"2022-09-20T01:31:28.844000Z","created_at":"2022-09-20T01:31:28.844000Z"},{"_id":"632917f0a6ecce368d5d2945","body":"thanks a lot!!!!\r\n\r\nI have a further question about multi target variables.\r\n\r\nI'm trying to get output for multiple targets with Softmax.\r\n\r\nWhich means sum of the target's outputs should be 1.\r\n\r\nWhat should i do then?\r\n\r\nThanks again for your kindness :)","issue_id":1660140579668,"origin_id":1009595549,"user_origin_id":16810004,"create_time":1641876353,"update_time":1641876353,"id":1663637488847,"updated_at":"2022-09-20T01:31:28.846000Z","created_at":"2022-09-20T01:31:28.846000Z"},{"_id":"632917f0a6ecce368d5d2947","body":"> thanks a lot!!!!\r\n> \r\n> I have a further question about multi target variables.\r\n> \r\n> I'm trying to get output for multiple targets with Softmax.\r\n> \r\n> Which means sum of the target's outputs should be 1.\r\n> \r\n> What should i do then?\r\n> \r\n> Thanks again for your kindness :)\r\n\r\nWere you able to figure this out? I am running into issues with multiple targets, and now I'm stuck at the trainer.fit step. I am working on a multi-class classification problem. I have 7 different target variables and would like the sum of their probabilities -- coupled with ( 1 - (the probabilities of all 7 added together)) -- to add to 1. I followed the suggestions in this thread (MultiLoss and output size as a list). The output is below:\r\n\r\n`LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\r\n\r\n | Name | Type | Params\r\n----------------------------------------------------------------------------------------\r\n0 | loss | MultiLoss | 0 \r\n1 | logging_metrics | ModuleList | 0 \r\n2 | input_embeddings | MultiEmbedding | 1 \r\n3 | prescalers | ModuleDict | 336 \r\n4 | static_variable_selection | VariableSelectionNetwork | 10.8 K\r\n5 | encoder_variable_selection | VariableSelectionNetwork | 3.7 K \r\n6 | decoder_variable_selection | VariableSelectionNetwork | 528 \r\n7 | static_context_variable_selection | GatedResidualNetwork | 1.1 K \r\n8 | static_context_initial_hidden_lstm | GatedResidualNetwork | 1.1 K \r\n9 | static_context_initial_cell_lstm | GatedResidualNetwork | 1.1 K \r\n10 | static_context_enrichment | GatedResidualNetwork | 1.1 K \r\n11 | lstm_encoder | LSTM | 2.2 K \r\n12 | lstm_decoder | LSTM | 2.2 K \r\n13 | post_lstm_gate_encoder | GatedLinearUnit | 544 \r\n14 | post_lstm_add_norm_encoder | AddNorm | 32 \r\n15 | static_enrichment | GatedResidualNetwork | 1.4 K \r\n16 | multihead_attn | InterpretableMultiHeadAttention | 1.1 K \r\n17 | post_attn_gate_norm | GateAddNorm | 576 \r\n18 | pos_wise_ff | GatedResidualNetwork | 1.1 K \r\n19 | pre_output_gate_norm | GateAddNorm | 576 \r\n20 | output_layer | ModuleList | 952 \r\n----------------------------------------------------------------------------------------\r\n30.1 K Trainable params\r\n0 Non-trainable params\r\n30.1 K Total params\r\n0.120 Total estimated model params size (MB)\r\nSanity Checking DataLoader 0: 0%\r\n0\/1 [00:01<?, ?it\/s]\r\n---------------------------------------------------------------------------\r\nTypeError Traceback (most recent call last)\r\nInput In [45], in <cell line: 2>()\r\n 1 # fit network\r\n----> 2 trainer.fit(\r\n 3 tft,\r\n 4 train_dataloaders=train_dataloader,\r\n 5 val_dataloaders=val_dataloader,\r\n 6 )\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/trainer\/trainer.py:768, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)\r\n 749 r\"\"\"\r\n 750 Runs the full optimization routine.\r\n 751 \r\n (...)\r\n 765 datamodule: An instance of :class:`~pytorch_lightning.core.datamodule.LightningDataModule`.\r\n 766 \"\"\"\r\n 767 self.strategy.model = model\r\n--> 768 self._call_and_handle_interrupt(\r\n 769 self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path\r\n 770 )\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/trainer\/trainer.py:721, in Trainer._call_and_handle_interrupt(self, trainer_fn, *args, **kwargs)\r\n 719 return self.strategy.launcher.launch(trainer_fn, *args, trainer=self, **kwargs)\r\n 720 else:\r\n--> 721 return trainer_fn(*args, **kwargs)\r\n 722 # TODO: treat KeyboardInterrupt as BaseException (delete the code below) in v1.7\r\n 723 except KeyboardInterrupt as exception:\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/trainer\/trainer.py:809, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)\r\n 805 ckpt_path = ckpt_path or self.resume_from_checkpoint\r\n 806 self._ckpt_path = self.__set_ckpt_path(\r\n 807 ckpt_path, model_provided=True, model_connected=self.lightning_module is not None\r\n 808 )\r\n--> 809 results = self._run(model, ckpt_path=self.ckpt_path)\r\n 811 assert self.state.stopped\r\n 812 self.training = False\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/trainer\/trainer.py:1234, in Trainer._run(self, model, ckpt_path)\r\n 1230 self._checkpoint_connector.restore_training_state()\r\n 1232 self._checkpoint_connector.resume_end()\r\n-> 1234 results = self._run_stage()\r\n 1236 log.detail(f\"{self.__class__.__name__}: trainer tearing down\")\r\n 1237 self._teardown()\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/trainer\/trainer.py:1321, in Trainer._run_stage(self)\r\n 1319 if self.predicting:\r\n 1320 return self._run_predict()\r\n-> 1321 return self._run_train()\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/trainer\/trainer.py:1343, in Trainer._run_train(self)\r\n 1340 self._pre_training_routine()\r\n 1342 with isolate_rng():\r\n-> 1343 self._run_sanity_check()\r\n 1345 # enable train mode\r\n 1346 self.model.train()\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/trainer\/trainer.py:1411, in Trainer._run_sanity_check(self)\r\n 1409 # run eval step\r\n 1410 with torch.no_grad():\r\n-> 1411 val_loop.run()\r\n 1413 self._call_callback_hooks(\"on_sanity_check_end\")\r\n 1415 # reset logger connector\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/loops\/base.py:204, in Loop.run(self, *args, **kwargs)\r\n 202 try:\r\n 203 self.on_advance_start(*args, **kwargs)\r\n--> 204 self.advance(*args, **kwargs)\r\n 205 self.on_advance_end()\r\n 206 self._restarting = False\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/loops\/dataloader\/evaluation_loop.py:154, in EvaluationLoop.advance(self, *args, **kwargs)\r\n 152 if self.num_dataloaders > 1:\r\n 153 kwargs[\"dataloader_idx\"] = dataloader_idx\r\n--> 154 dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)\r\n 156 # store batch level output per dataloader\r\n 157 self._outputs.append(dl_outputs)\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/loops\/base.py:204, in Loop.run(self, *args, **kwargs)\r\n 202 try:\r\n 203 self.on_advance_start(*args, **kwargs)\r\n--> 204 self.advance(*args, **kwargs)\r\n 205 self.on_advance_end()\r\n 206 self._restarting = False\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/loops\/epoch\/evaluation_epoch_loop.py:127, in EvaluationEpochLoop.advance(self, data_fetcher, dl_max_batches, kwargs)\r\n 124 self.batch_progress.increment_started()\r\n 126 # lightning module methods\r\n--> 127 output = self._evaluation_step(**kwargs)\r\n 128 output = self._evaluation_step_end(output)\r\n 130 self.batch_progress.increment_processed()\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/loops\/epoch\/evaluation_epoch_loop.py:222, in EvaluationEpochLoop._evaluation_step(self, **kwargs)\r\n 220 output = self.trainer._call_strategy_hook(\"test_step\", *kwargs.values())\r\n 221 else:\r\n--> 222 output = self.trainer._call_strategy_hook(\"validation_step\", *kwargs.values())\r\n 224 return output\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/trainer\/trainer.py:1763, in Trainer._call_strategy_hook(self, hook_name, *args, **kwargs)\r\n 1760 return\r\n 1762 with self.profiler.profile(f\"[Strategy]{self.strategy.__class__.__name__}.{hook_name}\"):\r\n-> 1763 output = fn(*args, **kwargs)\r\n 1765 # restore current_fx when nested context\r\n 1766 pl_module._current_fx_name = prev_fx_name\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_lightning\/strategies\/strategy.py:344, in Strategy.validation_step(self, *args, **kwargs)\r\n 339 \"\"\"The actual validation step.\r\n 340 \r\n 341 See :meth:`~pytorch_lightning.core.lightning.LightningModule.validation_step` for more details\r\n 342 \"\"\"\r\n 343 with self.precision_plugin.val_step_context():\r\n--> 344 return self.model.validation_step(*args, **kwargs)\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_forecasting\/models\/base_model.py:413, in BaseModel.validation_step(self, batch, batch_idx)\r\n 411 def validation_step(self, batch, batch_idx):\r\n 412 x, y = batch\r\n--> 413 log, out = self.step(x, y, batch_idx)\r\n 414 log.update(self.create_log(x, y, out, batch_idx))\r\n 415 return log\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_forecasting\/models\/base_model.py:547, in BaseModel.step(self, x, y, batch_idx, **kwargs)\r\n 545 loss = loss * (1 + monotinicity_loss)\r\n 546 else:\r\n--> 547 out = self(x, **kwargs)\r\n 549 # calculate loss\r\n 550 prediction = out[\"prediction\"]\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/torch\/nn\/modules\/module.py:1110, in Module._call_impl(self, *input, **kwargs)\r\n 1106 # If we don't have any hooks, we want to skip the rest of the logic in\r\n 1107 # this function, and just call forward.\r\n 1108 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks\r\n 1109 or _global_forward_hooks or _global_forward_pre_hooks):\r\n-> 1110 return forward_call(*input, **kwargs)\r\n 1111 # Do not call functions when jit is used\r\n 1112 full_backward_hooks, non_full_backward_hooks = [], []\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_forecasting\/models\/temporal_fusion_transformer\/__init__.py:503, in TemporalFusionTransformer.forward(self, x)\r\n 499 else:\r\n 500 output = self.output_layer(output)\r\n 502 return self.to_network_output(\r\n--> 503 prediction=self.transform_output(output, target_scale=x[\"target_scale\"]),\r\n 504 encoder_attention=attn_output_weights[..., :max_encoder_length],\r\n 505 decoder_attention=attn_output_weights[..., max_encoder_length:],\r\n 506 static_variables=static_variable_selection,\r\n 507 encoder_variables=encoder_sparse_weights,\r\n 508 decoder_variables=decoder_sparse_weights,\r\n 509 decoder_lengths=decoder_lengths,\r\n 510 encoder_lengths=encoder_lengths,\r\n 511 )\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_forecasting\/models\/base_model.py:336, in BaseModel.transform_output(self, prediction, target_scale)\r\n 325 \"\"\"\r\n 326 Extract prediction from network output and rescale it to real space \/ de-normalize it.\r\n 327 \r\n (...)\r\n 333 torch.Tensor: rescaled prediction\r\n 334 \"\"\"\r\n 335 if isinstance(self.loss, MultiLoss):\r\n--> 336 out = self.loss.rescale_parameters(\r\n 337 prediction,\r\n 338 target_scale=target_scale,\r\n 339 encoder=self.output_transformer.normalizers, # need to use normalizer per encoder\r\n 340 )\r\n 341 else:\r\n 342 out = self.loss.rescale_parameters(prediction, target_scale=target_scale, encoder=self.output_transformer)\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_forecasting\/metrics.py:452, in MultiLoss.__getattr__.<locals>.func(*args, **kwargs)\r\n 438 new_args = [\r\n 439 arg[idx]\r\n 440 if isinstance(arg, (list, tuple))\r\n (...)\r\n 444 for arg in args\r\n 445 ]\r\n 446 new_kwargs = {\r\n 447 key: val[idx]\r\n 448 if isinstance(val, list) and not isinstance(val, rnn.PackedSequence) and len(val) == n\r\n 449 else val\r\n 450 for key, val in kwargs.items()\r\n 451 }\r\n--> 452 results.append(getattr(m, name)(*new_args, **new_kwargs))\r\n 453 return results\r\n\r\nFile ~\/.local\/lib\/python3.8\/site-packages\/pytorch_forecasting\/metrics.py:75, in Metric.rescale_parameters(self, parameters, target_scale, encoder)\r\n 61 def rescale_parameters(\r\n 62 self, parameters: torch.Tensor, target_scale: torch.Tensor, encoder: BaseEstimator\r\n 63 ) -> torch.Tensor:\r\n 64 \"\"\"\r\n 65 Rescale normalized parameters into the scale required for the output.\r\n 66 \r\n (...)\r\n 73 torch.Tensor: parameters in real\/not normalized space\r\n 74 \"\"\"\r\n---> 75 return encoder(dict(prediction=parameters, target_scale=target_scale))\r\n\r\nTypeError: 'list' object is not callable`","issue_id":1660140579668,"origin_id":1137898515,"user_origin_id":42918800,"create_time":1653517202,"update_time":1653517202,"id":1663637488851,"updated_at":"2022-09-20T01:31:28.850000Z","created_at":"2022-09-20T01:31:28.850000Z"}]
- PyTorch-Forecasting version: 0.9.2 - PyTorch version: - PyTorch-Lightning: 1.5.7 - Python version: 3.8.9 - Operating System: Win10 I'm trying to create a dataset with multiple targets. In my case...
AssertionError: filters should not remove entries all entries - check encoder/decoder lengths and lags for TFT
[{"_id":"632915e30d68a95f041c6973","body":"Could it be that the error occurs because you are grouping by the variable \"AMOUNT\"? I think this will group all your entries which have the same amount value and results in groups existing of only one row. Hence, the encoder\/decoder length is larger than your group. As I understand it correctly, you only have one time series, so add a constant column and use it as a grouping variable. Also take care that your time index shall increase with the date points and not include some duplicates in the same time series. ","issue_id":1660140579671,"origin_id":1136760529,"user_origin_id":57753212,"create_time":1653458190,"update_time":1653458190,"id":1663636963080,"updated_at":"2022-09-20T01:22:43.080000Z","created_at":"2022-09-20T01:22:43.080000Z"},{"_id":"632915e30d68a95f041c6974","body":"thanks for quick reponse and check with your suggestions and i will give you update by eod , thanks once again @[DominikPKaiser]","issue_id":1660140579671,"origin_id":1136820808,"user_origin_id":85454586,"create_time":1653461304,"update_time":1653461304,"id":1663636963109,"updated_at":"2022-09-20T01:22:43.108000Z","created_at":"2022-09-20T01:22:43.108000Z"}]
This is my data set having shape(208,3) there data column data start form 2015-12-25 to 2017-01-11 (having 14months of data ) This is the following settings i used. max_prediction_length =...
- PyTorch-Forecasting version: 0.10.1 - PyTorch version: 1.11.0+cu113 - Python version: 3.7.13 - Operating System: ### Expected behavior I tried to execute the sample code of N-HiTS tutorial on Colab....
Holidays effect
[{"_id":"632914f9a6ecce368d5d28e3","body":"Not exactly sure I understand your question, but let me try to answer: You should use the \"special_days\" variable as a known categorical variable. Could you post your definition of the TimeSeriesDataSet in addition to a plot of the variable importances in the decoder?","issue_id":1660140579675,"origin_id":864391330,"user_origin_id":13008540,"create_time":1624100744,"update_time":1624100744,"id":1663636729246,"updated_at":"2022-09-20T01:18:49.246000Z","created_at":"2022-09-20T01:18:49.246000Z"},{"_id":"632914f9a6ecce368d5d28e4","body":"Thanks for your response.\r\n\r\nThe special days are the fields `spring_festival`, `labour_day` and `national_day`. The structure of my data like the following:\r\n```\r\n df.head() date plants flow temperature_max temperature_min aqi \\\r\n0 2017-01-01 YSP 83.60 16 10 54\r\n1 2017-01-02 YSP 82.00 15 8 92\r\n2 2017-01-03 YSP 83.34 14 9 131\r\n3 2017-01-04 YSP 81.42 15 11 75\r\n4 2017-01-05 YSP 78.25 12 11 44\r\n\r\n spring_festival labour_day national_day time_idx dayofweek weather_type \\\r\n0 - - - 0 6 sunny\r\n1 - - - 1 0 cloudy\r\n2 - - - 2 1 sunny\r\n3 - - - 3 2 rainny\r\n4 - - - 4 3 rainny\r\n\r\n wind_type wind_force_type\r\n0 wind_south small_wind\r\n1 wind_north small_wind\r\n2 wind_others small_wind\r\n3 wind_others small_wind\r\n4 wind_others small_wind\r\n\r\n df.tail()\r\n date plants flow temperature_max temperature_min aqi \\\r\n17793 2021-06-02 ZJ 387.04 22 20 32\r\n17794 2021-06-03 ZJ 402.11 27 20 59\r\n17795 2021-06-04 ZJ 401.26 26 20 50\r\n17796 2021-06-05 ZJ 412.83 32 21 61\r\n17797 2021-06-06 ZJ 418.75 33 22 71\r\n\r\n spring_festival labour_day national_day time_idx dayofweek \\\r\n17793 - - - 1612 2\r\n17794 - - - 1613 3\r\n17795 - - - 1614 4\r\n17796 - - - 1615 5\r\n17797 - - - 1616 6\r\n\r\n weather_type wind_type wind_force_type\r\n17793 rainny wind_south large_wind\r\n17794 cloudy wind_north small_wind\r\n17795 cloudy wind_south small_wind\r\n17796 sunny wind_south small_wind\r\n17797 sunny wind_south small_wind\r\n\r\ndf.describe()\r\n flow temperature_max temperature_min aqi \\\r\ncount 17798.000000 17798.000000 17798.000000 17798.000000\r\nmean 74.589377 21.121755 14.987021 58.069221\r\nstd 110.725732 8.862504 8.256358 28.449594\r\nmin 0.000000 -3.000000 -6.000000 15.000000\r\n25% 16.240000 13.000000 8.000000 38.000000\r\n50% 36.600000 22.000000 15.000000 50.000000\r\n75% 73.977500 28.000000 22.000000 72.000000\r\nmax 487.490000 40.000000 31.000000 248.000000\r\n\r\n time_idx\r\ncount 17798.000000\r\nmean 808.402967\r\nstd 466.937707\r\nmin 0.000000\r\n25% 404.000000\r\n50% 808.500000\r\n75% 1213.000000\r\nmax 1616.000000\r\n```\r\nAnd the codes:\r\n**I found if the `min_encoder_length` was set to be equal to the `max_encoder_length`, like the value 60, the training process would need more than 10 hours for my Lenovo Thinkpad X1 Carbon. And if the `min_encoder_length` is less than the `max_encoder_length`, the training time will be less than half an hour. Why?**\r\n```python\r\ntraining_cutoff = 365 * 3\r\nspecial_days = ['spring_festival', 'labour_day', 'national_day']\r\ntraining = TimeSeriesDataSet(\r\n train_df[lambda x: x.time_idx <= training_cutoff],\r\n time_idx=\"time_idx\",\r\n target=\"flow\",\r\n group_ids=[\"plants\"],\r\n min_encoder_length=60, \r\n max_encoder_length=90,\r\n min_prediction_length=7,\r\n max_prediction_length=7,\r\n static_categoricals=[],\r\n static_reals=[],\r\n time_varying_known_categoricals=[\r\n \"special_days\",\\\r\n \"dayofweek\", \\\r\n \"weather_type\", \"wind_type\", \"wind_force_type\"],\r\n variable_groups={\"special_days\": special_days}, # group of categorical variables can be treated as one variable\r\n time_varying_known_reals=[\r\n \"temperature_max\",\r\n \"temperature_min\"],\r\n time_varying_unknown_categoricals=[],\r\n time_varying_unknown_reals=[\"flow\"],\r\n #lags={\"flow\":[1]},\r\n target_normalizer=GroupNormalizer(\r\n groups=[\"plants\"], transformation=\"softplus\"\r\n ), # use softplus and normalize by group\r\n add_relative_time_idx=True,\r\n add_target_scales=True,\r\n add_encoder_length=True,\r\n allow_missings=True,\r\n)\r\n# create validation set (predict=True) which means to predict the last max_prediction_length points in time\r\n# for each series\r\nvalidation = TimeSeriesDataSet.from_dataset(\r\n training, \r\n train_df[lambda x: x.time_idx > training_cutoff],\r\n# predict = True,\r\n stop_randomization=True)\r\ntest = TimeSeriesDataSet.from_dataset(\r\n training, \r\n test_df,\r\n# predict = True,\r\n stop_randomization=True)\r\n# create dataloaders for model\r\nbatch_size = 32 # set this between 32 to 128\r\ntrain_dataloader = training.to_dataloader(train=True, batch_size=batch_size, num_workers=0)\r\nval_dataloader = validation.to_dataloader(train=False, batch_size=batch_size * 10, num_workers=0)\r\ntest_dataloader = test.to_dataloader(train=False, batch_size=batch_size * 10, num_workers=0)\r\n\r\n##\r\n#Train the Temporal Fusion Transformer\r\n\r\n# ### Find optimal learning rate\r\npl.seed_everything(0)\r\ntrainer = pl.Trainer(\r\n gpus=0,\r\n # clipping gradients is a hyperparameter and important to prevent divergance\r\n # of the gradient for recurrent neural networks\r\n gradient_clip_val=0.1,\r\n)\r\n\r\n\r\ntft = TemporalFusionTransformer.from_dataset(\r\n training,\r\n # not meaningful for finding the learning rate but otherwise very important\r\n learning_rate=0.03,\r\n hidden_size=16, # most important hyperparameter apart from learning rate\r\n # number of attention heads. Set to up to 4 for large datasets\r\n attention_head_size=1,\r\n dropout=0.1, # between 0.1 and 0.3 are good values\r\n hidden_continuous_size=8, # set to <= hidden_size\r\n output_size=7, # 7 quantiles by default\r\n loss=QuantileLoss(),\r\n # reduce learning rate if no improvement in validation loss after x epochs\r\n reduce_on_plateau_patience=4,\r\n)\r\nprint(f\"Number of parameters in network: {tft.size()\/1e3:.1f}k\")\r\n# ### Train model\r\n\r\n# configure network and trainer\r\nearly_stop_callback = EarlyStopping(monitor=\"val_loss\", min_delta=1e-4, patience=10, verbose=False, mode=\"min\")\r\nlr_logger = LearningRateMonitor() # log the learning rate\r\nlogger = TensorBoardLogger(\"lightning_logs\") # logging results to a tensorboard\r\n\r\ntrainer = pl.Trainer(\r\n max_epochs=30,\r\n gpus=0,\r\n weights_summary=\"top\",\r\n gradient_clip_val=0.1,\r\n limit_train_batches=30, # coment in for training, running valiation every 30 batches\r\n # fast_dev_run=True, # comment in to check that networkor dataset has no serious bugs\r\n callbacks=[lr_logger, early_stop_callback],\r\n logger=logger,\r\n)\r\n\r\n\r\ntft = TemporalFusionTransformer.from_dataset(\r\n training,\r\n learning_rate=0.001,\r\n hidden_size=16,\r\n attention_head_size=1,\r\n dropout=0.1,\r\n hidden_continuous_size=16,\r\n output_size=7, # 7 quantiles by default\r\n loss=QuantileLoss(),\r\n log_interval=10, # uncomment for learning rate finder and otherwise, e.g. to 10 for logging every 10 batches\r\n reduce_on_plateau_patience=4,\r\n)\r\nprint(f\"Number of parameters in network: {tft.size()\/1e3:.1f}k\")\r\n\r\n##\r\n# fit network\r\ntrainer.fit(\r\n tft,\r\n train_dataloader=train_dataloader,\r\n val_dataloaders=val_dataloader,\r\n)\r\n##\r\n# Specify a path\r\nPATH = \"..\/model\/all_plants_2_7_entire_model.pt\"\r\n# Save\r\ntorch.save(trainer, PATH)\r\n# Load\r\nmodel = torch.load(PATH)\r\n##\r\n# load the best model according to the validation loss\r\n# (given that we use early stopping, this is not necessarily the last epoch)\r\nbest_model_path = model.checkpoint_callback.best_model_path\r\nbest_tft = TemporalFusionTransformer.load_from_checkpoint(best_model_path)\r\n# feature importants\r\nprediction_tft_raw, pred_idx_df = best_tft.predict(test_dataloader, mode=\"raw\", return_index = True)\r\ninterpretation = best_tft.interpret_output(prediction_tft_raw, reduction=\"sum\")\r\nfigs_imp = best_tft.plot_interpretation(interpretation)\r\n```\r\n**I don't understand why the following error produced by the `raw` predict .**\r\n```\r\nprediction_tft_raw, pred_idx_df = best_tft.predict(test_dataloader, mode=\"raw\", return_ ...: index = True) --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-92-4a2ba89f2e3c> in <module> ----> 1 prediction_tft_raw, pred_idx_df = best_tft.predict(test_dataloader, mode=\"raw\", return_index = True) ~\/.local\/lib\/python3.6\/site-packages\/pytorch_forecasting\/models\/base_model.py in predict(self, data, mode, return_index, return_decoder_lengths, batch_size, num_workers, fast_dev_run, show_progress_bar, return_x, mode_kwargs, **kwargs) 1058 output = _torch_cat_na(output) 1059 elif mode == \"raw\": -> 1060 output = _concatenate_output(output) 1061 1062 # generate output\r\n\r\n~\/.local\/lib\/python3.6\/site-packages\/pytorch_forecasting\/models\/base_model.py in _concatenate_output(output)\r\n 90 # concatenate simple tensors\r\n 91 if isinstance(v0, torch.Tensor):\r\n---> 92 output_cat[name] = _torch_cat_na([out[name] for out in output])\r\n 93 # concatenate list of tensors\r\n 94 elif isinstance(v0, (tuple, list)) and len(v0) > 0:\r\n~\/.local\/lib\/python3.6\/site-packages\/pytorch_forecasting\/models\/base_model.py in _torch_cat_na(x)\r\n 68 for xi in x\r\n 69 ]\r\n---> 70 return torch.cat(x, dim=0)\r\n 71\r\n 72\r\nRuntimeError: Sizes of tensors must match except in dimension 0. Got 98 and 97 in dimension 3 (The offending index is 5)\r\n```","issue_id":1660140579675,"origin_id":864494762,"user_origin_id":9900304,"create_time":1624160300,"update_time":1624160300,"id":1663636729251,"updated_at":"2022-09-20T01:18:49.250000Z","created_at":"2022-09-20T01:18:49.250000Z"},{"_id":"632914f9a6ecce368d5d28e6","body":"same error","issue_id":1660140579675,"origin_id":864820132,"user_origin_id":47316871,"create_time":1624262690,"update_time":1624262690,"id":1663636729256,"updated_at":"2022-09-20T01:18:49.256000Z","created_at":"2022-09-20T01:18:49.256000Z"},{"_id":"632914f9a6ecce368d5d28e7","body":"@zhangxin0611 I met the same error, have you solved it?","issue_id":1660140579675,"origin_id":962788345,"user_origin_id":2903328,"create_time":1636343782,"update_time":1636343782,"id":1663636729261,"updated_at":"2022-09-20T01:18:49.261000Z","created_at":"2022-09-20T01:18:49.261000Z"},{"_id":"632914f9a6ecce368d5d28e8","body":"no, and would be interested in hearing any progress...\n\nOn Mon, Nov 8, 2021 at 5:56 AM Adam ***@***.***> wrote:\n\n> @zhangxin0611 <https:\/\/github.com\/zhangxin0611> I met the same error,\n> have you solved it?\n>\n> \u2014\n> You are receiving this because you commented.\n> Reply to this email directly, view it on GitHub\n> <https:\/\/github.com\/jdb78\/pytorch-forecasting\/issues\/565#issuecomment-962788345>,\n> or unsubscribe\n> <https:\/\/github.com\/notifications\/unsubscribe-auth\/ALI77B2HY5Y7W3C5JT7STJLUK5C7DANCNFSM46WGONOA>\n> .\n> Triage notifications on the go with GitHub Mobile for iOS\n> <https:\/\/apps.apple.com\/app\/apple-store\/id1477376905?ct=notification-email&mt=8&pt=524675>\n> or Android\n> <https:\/\/play.google.com\/store\/apps\/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.\n>\n>\n","issue_id":1660140579675,"origin_id":982422110,"user_origin_id":47316871,"create_time":1638262856,"update_time":1638262856,"id":1663636729264,"updated_at":"2022-09-20T01:18:49.264000Z","created_at":"2022-09-20T01:18:49.264000Z"},{"_id":"632914f9a6ecce368d5d28e9","body":"Do you have a reproducible example with data? I just cannot reproduce the error.","issue_id":1660140579675,"origin_id":983711716,"user_origin_id":13008540,"create_time":1638369952,"update_time":1638369952,"id":1663636729267,"updated_at":"2022-09-20T01:18:49.267000Z","created_at":"2022-09-20T01:18:49.267000Z"},{"_id":"632914f9a6ecce368d5d28ea","body":"I'm working on this project again - my notebooks can be reached through the\ncomment here as well:\nhttps:\/\/github.com\/jdb78\/pytorch-forecasting\/issues\/579\n\nCode to reproduce the problem\n\nHere is a link to a colab notebook to run the code and reproduce the\noutput: https:\/\/github.com\/Bassie1\/notebooks\/blob\/main\/model_github.ipynb\nAnd here is a link to the dataset used:\nhttps:\/\/github.com\/Bassie1\/notebooks\/blob\/main\/github_data.csv\n\nOn Wed, Dec 1, 2021 at 4:46 PM Jan Beitner ***@***.***> wrote:\n\n> Do you have a reproducible example with data? I just cannot reproduce the\n> error.\n>\n> \u2014\n> You are receiving this because you commented.\n> Reply to this email directly, view it on GitHub\n> <https:\/\/github.com\/jdb78\/pytorch-forecasting\/issues\/565#issuecomment-983711716>,\n> or unsubscribe\n> <https:\/\/github.com\/notifications\/unsubscribe-auth\/ALI77BZAZMP34COFSSY3CI3UOYYKXANCNFSM46WGONOA>\n> .\n> Triage notifications on the go with GitHub Mobile for iOS\n> <https:\/\/apps.apple.com\/app\/apple-store\/id1477376905?ct=notification-email&mt=8&pt=524675>\n> or Android\n> <https:\/\/play.google.com\/store\/apps\/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.\n>\n>\n","issue_id":1660140579675,"origin_id":1135519876,"user_origin_id":47316871,"create_time":1653377935,"update_time":1653377935,"id":1663636729270,"updated_at":"2022-09-20T01:18:49.270000Z","created_at":"2022-09-20T01:18:49.270000Z"}]
I tried to use the pytorch-forecasting for about 10 days with my own data. I found the holidays effect without be grasped by the model though the special days setted...
question
- PyTorch-Forecasting version: 0.10.1 - PyTorch version: 1.10.2+cu111 - Python version: 3.8.10 - Operating System: Ubuntu I recently experienced a `dtype` issue when using `TorchNormalizer` and managed to identify the...
Softplus transformation really slow in group normalizer
[{"_id":"632914f00d68a95f041c6945","body":"Actually any transformation in the target normalizer (using group or encoder normalizer) makes the process really slow... Any thoughts on this? Is it really necessary\/important to use a target_normalizer?","issue_id":1660140579678,"origin_id":1122653374,"user_origin_id":27287222,"create_time":1652202323,"update_time":1652202323,"id":1663636720087,"updated_at":"2022-09-20T01:18:40.087000Z","created_at":"2022-09-20T01:18:40.087000Z"},{"_id":"632914f00d68a95f041c6946","body":"Yes, pretty important.","issue_id":1660140579678,"origin_id":1134428074,"user_origin_id":13008540,"create_time":1653298478,"update_time":1653298478,"id":1663636720091,"updated_at":"2022-09-20T01:18:40.090000Z","created_at":"2022-09-20T01:18:40.090000Z"},{"_id":"632914f00d68a95f041c6947","body":"Is it normal that it slows down the process really much? I am forecasting positive integers, so i would really like to use softplus or something like softplus, relu or log1p. Which one could be better\/faster?\r\n\r\nThanks for the answer @jdb78 ","issue_id":1660140579678,"origin_id":1134644116,"user_origin_id":27287222,"create_time":1653310659,"update_time":1653310659,"id":1663636720093,"updated_at":"2022-09-20T01:18:40.093000Z","created_at":"2022-09-20T01:18:40.093000Z"},{"_id":"632914f00d68a95f041c6948","body":"Should be all very fast. Probably the issue is somewhere else in moving devices and copies - would need some proper profiling to find out.","issue_id":1660140579678,"origin_id":1134702944,"user_origin_id":13008540,"create_time":1653313687,"update_time":1653313687,"id":1663636720096,"updated_at":"2022-09-20T01:18:40.095000Z","created_at":"2022-09-20T01:18:40.095000Z"}]
I have trained a TFT with multi-gpu using DDP strategy and the model is 24x slower if i use a group normalizer as target_normalizer of the TS dataset with "softplus"...
## 🐛 Bug code：https://pytorch-forecasting.readthedocs.io/en/latest/tutorials/stallion.html When max_epochs=1 and use cpu, it costs **9.6s** on win 10 and **382.79s** on linux. Using **AdvancedProfiler** from pytorch-lightning, there is difference in **SingleDeviceStrategy.validation_step** between platforms...
How to plot predictions using DeepAR
[{"_id":"632914f39db0c2379380d403","body":"have you tried like this? Somebody posted this in another issue:\r\n\r\n```\r\nfor idx in range(180,200): \r\n best_model.plot_prediction(x, raw_predictions, idx=idx, add_loss_to_title=True, quantiles_kwargs={'use_metric': False}, prediction_kwargs={'use_metric': False})\r\n```","issue_id":1660140579683,"origin_id":1125763557,"user_origin_id":10365760,"create_time":1652428615,"update_time":1652428615,"id":1663636723197,"updated_at":"2022-09-20T01:18:43.197000Z","created_at":"2022-09-20T01:18:43.197000Z"}]
- PyTorch-Forecasting version: - PyTorch version: 0.10.1 - Python version: 3.8.10 - Operating System: Windows 10 ### Expected behaviour I ran the training for DeepAR, and I was expecting when...
- PyTorch-Forecasting version: 0.10.1 - PyTorch version: 1.11.0+cu113 - Python version: 3.7.13 - Operating System: VERSION="18.04.5 LTS (Bionic Beaver)" VERSION_ID="18.04" VERSION_CODENAME=bionic ### Expected behavior I executed code to fit my...
Failed to work on toy problem with a Saw function
[{"_id":"632915bda49e0e1da50b7677","body":"`limit_train_batches=30, # coment in for training, running valiation every 30 batches`\r\nfrom my understanding this results in aborting training after having seen the first few batches, so uncomment and see if it works better?","issue_id":1660140579688,"origin_id":1125783543,"user_origin_id":5399476,"create_time":1652430039,"update_time":1652430039,"id":1663636925016,"updated_at":"2022-09-20T01:22:05.015000Z","created_at":"2022-09-20T01:22:05.015000Z"}]
I adapted the Stallion demo to work with a really [simple toy dataset](https://github.com/ReddSpark1/pytorch/blob/main/pytorch%20forecasting%20test.ipynb) that plots a Saw function. I tried to get Pytorch forecasting to then predict it, but the...
