Specifying Loss Distributions for Temporal Fusion Transform

[esbraun]

Description

Currently the mxnet temporal fusion transformer only allows for quantile loss. It would be helpful to be able to specify other distributions (i.e. negative binomial for count data).

Would this change be possible?

References

The library pytorch_forecasting allows specification of the loss for its temporal fusion transformer implementation.

[kashif]

note I have such a version for pytorch: https://github.com/kashif/pytorch-transformer-ts/tree/main/tft

[esbraun]

Thank you! Is this something that you're planning on merging eventually as well?

[kashif]

yes eventually

[esbraun]

Update: I've empirically found significant improvement using the negative binomial distribution for a operations related count valued forecasting problem using the TFT implementation from kashif's repo over the mxnet TFT gluonts implementation and other gluonts algorithms. The implementation, however, lacks the full featured covariate support of the gluonts mxnet model. Hopefully the support for more loss functions could be wrapped into remediation for #2416.