Dtype issue in `TorchNormalizer(method="identity")`

[RonanFR]

• PyTorch-Forecasting version: 0.10.1
• PyTorch version: 1.10.2+cu111
• Python version: 3.8.10
• Operating System: Ubuntu

I recently experienced a dtype issue when using TorchNormalizer and managed to identify the source of the bug.

When TorchNormalizer(method="identity", center=False) is fitted on pandas DataFrame/Series, the attributes self.center_and self.scale_ are set to np.zeros(y_center.shape[:-1]) and np.ones(y_scale.shape[:-1]) respectively (lines 467 and 468 of file pytorch_forecasting.data.encoders.py).

The default dtype for a numpy array is np.float64 and so both self.center_and self.scale_ have this dtype independently of the dtype of y_center and y_scale. On the other hand, the default dtype of torch (unless specified otherwise using torch.set_default_dtype) is float32 (a.k.a. Float, as opposed to Double which is equivalent to float64).

This may cause problems at inference time (e.g., with DeepAR): the predicted values have type float64 while the targets in the dataloader have type float32 (due to scaling). More specifically, at some point during inference (method predict), the model calls the method self.decode_autoregressive that iteratively fills a torch tensor using method self.decode_one (line 292 of file pytorch_forecasting.models.deepar.py). The source tensor has type float32 while the other tensor has type float64.

My suggestion would be to modify the code of TorchNormalizer. One possibility would be to set the type of self.center_and self.scale_ to be the same as y_centerand y_scale respectively. The problem is that it may be a bit weird if the type of y_scale is int32 for instance (we expect the scale to be a real value rather than an integer). This happens when using NegativeBinomialLoss for instance. Alternatively, we could set the type of self.center_and self.scale_ to be equal to str(torch.get_default_dtype())[6:] (this is the current fix I made in my code but not sure it is the best idea).