TCN of pytorch-forecasting

[wangbingnan136]

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codecov-commenter]

Codecov Report

Merging #549 (d4bc616) into master (06dcc29) will decrease coverage by 0.02%. The diff coverage is 88.09%.

@@            Coverage Diff             @@
##           master     #549      +/-   ##
==========================================
- Coverage   89.12%   89.09%   -0.03%     
==========================================
  Files          24       25       +1     
  Lines        3788     3907     +119     
==========================================
+ Hits         3376     3481     +105     
- Misses        412      426      +14     

Flag	Coverage Δ
cpu	89.09% <88.09%> (-0.03%)	:arrow_down:
pytest	89.09% <88.09%> (-0.03%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pytorch_forecasting/__init__.py	100.00% <ø> (ø)
pytorch_forecasting/models/base_model.py	87.65% <58.33%> (-0.47%)	:arrow_down:
.../models/temporal_convolutional_network/__init__.py	91.07% <91.07%> (ø)
pytorch_forecasting/models/__init__.py	100.00% <100.00%> (ø)
pytorch_forecasting/models/baseline.py	94.73% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 06dcc29...d4bc616. Read the comment docs.

[wangbingnan136]

Ok,I will do that as soon as I can!

[wangbingnan136]

I have modified TCN according to your suggestion,thanks a lot!

[wangbingnan136]

I'm sorry.I do not know how the checks above works? Can you give me some complete examples of code testing?So that I can do the test myself.

[jdb78]

In the directory tests/test_models are multiple files that test a network. I suggest to pretty much copy paste the test_nbeats.py. as this is very similar to what you are trying to do.

[jdb78]

There seems to be an import error: https://github.com/jdb78/pytorch-forecasting/pull/549/checks?check_run_id=2783133836

[wangbingnan136]

I have fixed the import error

[jdb78]

@wangbingnan136: How do we approach the multi-horizon forecasting issue with TCNs? Is it supposed to be re-current or a multi-horizon forecast? In case of the later, it is a bit unclear to me how to leverage covariates in the decoder.

[wangbingnan136]

@wangbingnan136: How do we approach the multi-horizon forecasting issue with TCNs? Is it supposed to be re-current or a multi-horizon forecast? In case of the later, it is a bit unclear to me how to leverage covariates in the decoder.

Tcn can only support the output mode of vector output, It just like nbeats or simple LSTM.

which I think is what you mean "re-current," that is, TCN only has an encoder structure and no decoder structure. We use the latent vector obtained by the encoder to directly connect to a fully connected layer to complete the forecasting task.

If we want to approach the multi-horizon forecasting issue with TCNs,we can direct use Wavenet ,which supports the structure of encoder+decoder.

[jdb78]

@wangbingnan136 Could you check the implementation? I am not really sure it is correct. Adding a skip connection (optional in the paper) dramatically improved the performance. Could you check it and confirm the implementation is correct before we merge?

[jdb78]

I think there are some changes required when I compare with https://www.mdpi.com/2079-9292/8/8/876

[wangbingnan136]

Ok,I will see

[wangbingnan136]

I have implemented the skip connection function according to https://github.com/philipperemy/keras-tcn/blob/master/tcn/tcn.py

and test the data in tutorial:

The performance of model becomes better.

[jdb78]

@wangbingnan136 , could you revert the version to after the lock-file fix (best to use git pull before continuing the work) and use the pre-commit hooks?

I believe the main issue is that M-TCN uses a slightly modified architecture and currently, the model uses convolutions along features and not time.

[wangbingnan136]

@wangbingnan136 , could you revert the version to after the lock-file fix (best to use git pull before continuing the work) and use the pre-commit hooks?

I believe the main issue is that M-TCN uses a slightly modified architecture and currently, the model uses convolutions along features and not time.

Ok,I have reverted the version.

[wangbingnan136]

I think the model uses convolutions along time.

24 is the length of time steps

[StatMixedML]

@wangbingnan136 Many thanks for making the TCN model available. However, I am not sure which version of the following you are actually implementing:

• Probabilistic forecasting with temporal convolutional neural network: Preferred one since this outputs and trains on quantile losses. Code available here.
• A Multi-Horizon Quantile Recurrent Forecaster: Available in GluonTS here.
• Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting.
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. Code available here.

May I ask you to be more specific on which one you are using. I assume it is the last one?

[wangbingnan136]

@wangbingnan136 Many thanks for making the TCN model available. However, I am not sure which version of the following you are actually implementing:

• Probabilistic forecasting with temporal convolutional neural network: Preferred one since this outputs and trains on quantile losses
• A Multi-Horizon Quantile Recurrent Forecaster: Available in GluonTS
• Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

May I ask you to be more specific on which one you are using.

Actually it is almost the same as https://github.com/philipperemy/keras-tcn. I think the related paper is this one:https://arxiv.org/pdf/1803.01271.pdf 《An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling》

[StatMixedML]

Actually it is almost the same as https://github.com/philipperemy/keras-tcn. I think the related paper is this one:https://arxiv.org/pdf/1803.01271.pdf 《An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling》

Ok I see, many thanks for clarifying!

[wangbingnan136]

Actually it is almost the same as https://github.com/philipperemy/keras-tcn. I think the related paper is this one:https://arxiv.org/pdf/1803.01271.pdf 《An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling》

Ok I see, many thanks for clarifying!

You are welcome~

[jdb78]

Actually, seeing this, we should probably switch to Probabilistic forecasting with temporal convolutional neural network as it has a proper decoder (The quantile loss is trivial and would be supported anyways).

[wangbingnan136]

Actually, seeing this, we should probably switch to Probabilistic forecasting with temporal convolutional neural network as it has a proper decoder (The quantile loss is trivial and would be supported anyways).

I see.I may need some time to study the network structure of the tcn in this paper。

[StatMixedML]

Actually, seeing this, we should probably switch to Probabilistic forecasting with temporal convolutional neural network as it has a proper decoder (The quantile loss is trivial and would be supported anyways).

@jdb78 I would also opt for this version. Would love to be part of supporting the implementation.