know and unknow values

[korosig]

• PyTorch-Forecasting version: 0.9.2
• PyTorch version: '1.10.1+cu113'
• Python version: 3.7
• Operating System: ubuntu

Expected behavior

I want to use the TimeSeriesDataSet function, the description says that the UNKNOW values represent the past and the KNOW values represent what we know in the past and the future.

Actual behavior

I did some values in a random test, and I pass feature '1' and '2' as KNOW values, even '3' as UNKNOW. But I can't interpret the result shown in the output. Both "encoder_cont" and "decoder_cont" see all values of all 3 features, although my idea is that the decoder should have one less parameter.

Code to reproduce the problem

data = np.array([[1,2,3,4,5,6,7,8,9,1,100,0],
                 [11,12,13,14,15,16,17,18,19,2,101,0],
                 [21,22,23,24,25,26,27,28,29,3,102,0],
                 [31,32,33,34,35,36,37,38,39,4,103,0],
                 [41,42,43,44,45,46,47,48,49,5,104,0],
                 [51,52,53,54,55,56,57,58,59,6,105,0],
                 [61,62,63,64,65,66,67,68,69,7,106,0],
                 [71,72,73,74,75,76,77,78,79,8,107,0],
                 [81,82,83,84,85,86,87,88,89,9,108,0]    
                    ])
data = pd.DataFrame(data)
data.columns = ['1','2','3','4','5','6','7','8','9','time_idx','target','NODE_ID']
data['NODE_ID'] = data['NODE_ID'].astype(str)

max_encoder_length = 4
max_prediction_length = 4

training = TimeSeriesDataSet(
    data,
    time_idx='time_idx',
    target="target",
    group_ids=['NODE_ID'],
    min_encoder_length=max_encoder_length, 
    max_encoder_length=max_encoder_length,
    min_prediction_length=max_prediction_length,
    max_prediction_length=max_prediction_length,
    time_varying_known_reals=['1','2'], 
    time_varying_unknown_reals=['3'],
    scalers= {'1': None,'2': None, '3': None}
)
train_dataloader = training.to_dataloader(train=True, batch_size=2, num_workers=0)

training.get_parameters()

{'time_idx': 'time_idx',
 'target': 'target',
 'group_ids': ['NODE_ID'],
 'weight': None,
 'max_encoder_length': 4,
 'min_encoder_length': 4,
 'min_prediction_idx': 1,
 'min_prediction_length': 4,
 'max_prediction_length': 4,
 'static_categoricals': [],
 'static_reals': [],
 'time_varying_known_categoricals': [],
 'time_varying_known_reals': ['1', '2'],
 'time_varying_unknown_categoricals': [],
 'time_varying_unknown_reals': ['3'],
 'variable_groups': {},
 'constant_fill_strategy': {},
 'allow_missing_timesteps': False,
 'lags': {},
 'add_relative_time_idx': False,
 'add_target_scales': False,
 'add_encoder_length': False,
 'target_normalizer': NaNLabelEncoder(),
 'categorical_encoders': {'__group_id__NODE_ID': NaNLabelEncoder()},
 'scalers': {'1': None, '2': None, '3': None},
 'randomize_length': None,
 'predict_mode': False}

x, y = next(iter(train_dataloader))
x
{'encoder_cat': tensor([], size=(2, 4, 0), dtype=torch.int64),
 'encoder_cont': tensor([[[11., 12., 13.],
          [21., 22., 23.],
          [31., 32., 33.],
          [41., 42., 43.]],
 
         [[ 1.,  2.,  3.],
          [11., 12., 13.],
          [21., 22., 23.],
          [31., 32., 33.]]]),
 'encoder_target': tensor([[1, 2, 3, 4],
         [0, 1, 2, 3]]),
 'encoder_lengths': tensor([4, 4]),
 'decoder_cat': tensor([], size=(2, 4, 0), dtype=torch.int64),
 'decoder_cont': tensor([[[51., 52., 53.],
          [61., 62., 63.],
          [71., 72., 73.],
          [81., 82., 83.]],
 
         [[41., 42., 43.],
          [51., 52., 53.],
          [61., 62., 63.],
          [71., 72., 73.]]]),
 'decoder_target': tensor([[5, 6, 7, 8],
         [4, 5, 6, 7]]),
 'decoder_lengths': tensor([4, 4]),
 'decoder_time_idx': tensor([[6, 7, 8, 9],
         [5, 6, 7, 8]]),
 'groups': tensor([[0],
         [0]]),
 'target_scale': tensor([[0., 0.],
         [0., 0.]])}

[korosig]

Actually, what I would like to do is to give a TFT model, for example, 10 features whose past we know and 3 features whose past and future we know. E.g. the prediction of the airplane passengers for the next 40 hours, where we know how many people were traveling per day, how many planes were in the airspace at the same time, how much was the price of a ticket (UNKNOW in the future) and we know when there are holidays, weekends, nice sunny forecast (KNOW in the future).

However, this data processing seems to me like all these features are processed as KNOW.

[Forecastlife]

I think am running into this issue as well. I am trying to use this function:

new_raw_predictions, new_x = best_tft.predict(new_prediction_data, mode="raw", return_x=False)

Based off of this data configuration for the TFT: training = TimeSeriesDataSet( data[lambda x: x.level_0 <= training_cutoff], time_idx="level_0", target="Ty_OnRoad_Del_Vol", group_ids=["ID"], min_encoder_length=max_encoder_length // 2, # keep encoder length long (as it is in the validation set) max_encoder_length=max_encoder_length, min_prediction_length=1, max_prediction_length=max_prediction_length, static_categoricals=[], static_reals=[], time_varying_known_categoricals=['Month_Num', 'Period_Day_Num', 'Day of Year', 'Week Num','Holiday'], variable_groups={}, # group of categorical variables can be treated as one variable time_varying_known_reals=['OR Deliv 1Y Lag', 'OR Deliv 1W Lag','OR Deliv 2W Lag'], time_varying_unknown_categoricals=[], time_varying_unknown_reals=[ 'Ty_OnRoad_Del_Vol', 'Ty_OnRoad_Pkp_Vol', 'Ty_Preload_Vol', 'Ty_Localsort_Vol', 'Ty_OnRoad_Stp_Missed_Del_Vol', 'Ty_OnRoad_Stp_Missed_Pkp_Vol', 'Ty_OnRoad_Send_Again_Vol', 'Ty_OnRoad_Late_Premium_Vol', 'Ty_OnRoad_Lib_Vol','ASB MRC', 'ENT MRC', 'SMB MRC', 'DOM MRC' ], target_normalizer=GroupNormalizer( groups=["ID"], transformation="softplus" ), # use softplus and normalize by group add_relative_time_idx=True, add_target_scales=True, add_encoder_length=True, )

But i am receiving this error: 61 (20%) of Ty_OnRoad_Del_Vol values were found to be NA or infinite (even after encoding) ((This is my prediction length/ decoder size) . NA values are not allowed allow_missing_timesteps refers to missing rows, not to missing values. Possible strategies to fix the issue are (a) dropping the variable Ty_OnRoad_Del_Vol, (b) using NaNLabelEncoder(add_nan=True) for categorical variables, (c) filling missing values and/or (d) optionally adding a variable indicating filled values

I have this value set in the time varying unknown's and as my target. Does anyone know why the decoder would need to have values in for these Vars? it was my understanding that time varying unknowns were for continuous vars that were known in the past but not the future.

My Encoder and Decoder datasets for the prediction space were concatenated on top of one another as done in the tutorial.

Thank you for your time if you read this!!

[korosig]

Ty_OnRoad_Del_Vol

In the meantime, I have come to understand how the Model works. When preparing data, all data must exist. But when the model is built (TemporalFusionTransformer.from_datase) the from_datase function (from class BaseModelWithCovariates) selects the variable names and creates the TFT based on them. So if you don't have data for the future (Ty_OnRoad_Del_Vol) then you can fill those values with some values e.g. fillna(-1). This has no meaning in the prediction, it's just that there is no NA in the array.

[Forecastlife]

Thank you sir! I ran a fill forward imputation and the forecasts weren't wildly incorrect so I thought this could be a possibility. Appreciate the help!

[LiuNull]

mark