pytorch-forecasting
pytorch-forecasting copied to clipboard
jdb78
Holidays effect
I tried to use the pytorch-forecasting for about 10 days with my own data. I found the holidays effect without be grasped by the model though the special days setted like the Demand forecasting with the Temporal Fusion Transformer. What parameters should be noticed for learning the holidays effect?
Thanks.
Not exactly sure I understand your question, but let me try to answer: You should use the "special_days" variable as a known categorical variable. Could you post your definition of the TimeSeriesDataSet in addition to a plot of the variable importances in the decoder?
Thanks for your response.
The special days are the fields spring_festival, labour_day and national_day. The structure of my data like the following:
df.head() date plants flow temperature_max temperature_min aqi \
0 2017-01-01 YSP 83.60 16 10 54
1 2017-01-02 YSP 82.00 15 8 92
2 2017-01-03 YSP 83.34 14 9 131
3 2017-01-04 YSP 81.42 15 11 75
4 2017-01-05 YSP 78.25 12 11 44
spring_festival labour_day national_day time_idx dayofweek weather_type \
0 - - - 0 6 sunny
1 - - - 1 0 cloudy
2 - - - 2 1 sunny
3 - - - 3 2 rainny
4 - - - 4 3 rainny
wind_type wind_force_type
0 wind_south small_wind
1 wind_north small_wind
2 wind_others small_wind
3 wind_others small_wind
4 wind_others small_wind
df.tail()
date plants flow temperature_max temperature_min aqi \
17793 2021-06-02 ZJ 387.04 22 20 32
17794 2021-06-03 ZJ 402.11 27 20 59
17795 2021-06-04 ZJ 401.26 26 20 50
17796 2021-06-05 ZJ 412.83 32 21 61
17797 2021-06-06 ZJ 418.75 33 22 71
spring_festival labour_day national_day time_idx dayofweek \
17793 - - - 1612 2
17794 - - - 1613 3
17795 - - - 1614 4
17796 - - - 1615 5
17797 - - - 1616 6
weather_type wind_type wind_force_type
17793 rainny wind_south large_wind
17794 cloudy wind_north small_wind
17795 cloudy wind_south small_wind
17796 sunny wind_south small_wind
17797 sunny wind_south small_wind
df.describe()
flow temperature_max temperature_min aqi \
count 17798.000000 17798.000000 17798.000000 17798.000000
mean 74.589377 21.121755 14.987021 58.069221
std 110.725732 8.862504 8.256358 28.449594
min 0.000000 -3.000000 -6.000000 15.000000
25% 16.240000 13.000000 8.000000 38.000000
50% 36.600000 22.000000 15.000000 50.000000
75% 73.977500 28.000000 22.000000 72.000000
max 487.490000 40.000000 31.000000 248.000000
time_idx
count 17798.000000
mean 808.402967
std 466.937707
min 0.000000
25% 404.000000
50% 808.500000
75% 1213.000000
max 1616.000000
And the codes:
I found if the min_encoder_length was set to be equal to the max_encoder_length, like the value 60, the training process would need more than 10 hours for my Lenovo Thinkpad X1 Carbon. And if the min_encoder_length is less than the max_encoder_length, the training time will be less than half an hour. Why?
training_cutoff = 365 * 3
special_days = ['spring_festival', 'labour_day', 'national_day']
training = TimeSeriesDataSet(
train_df[lambda x: x.time_idx <= training_cutoff],
time_idx="time_idx",
target="flow",
group_ids=["plants"],
min_encoder_length=60,
max_encoder_length=90,
min_prediction_length=7,
max_prediction_length=7,
static_categoricals=[],
static_reals=[],
time_varying_known_categoricals=[
"special_days",\
"dayofweek", \
"weather_type", "wind_type", "wind_force_type"],
variable_groups={"special_days": special_days}, # group of categorical variables can be treated as one variable
time_varying_known_reals=[
"temperature_max",
"temperature_min"],
time_varying_unknown_categoricals=[],
time_varying_unknown_reals=["flow"],
#lags={"flow":[1]},
target_normalizer=GroupNormalizer(
groups=["plants"], transformation="softplus"
), # use softplus and normalize by group
add_relative_time_idx=True,
add_target_scales=True,
add_encoder_length=True,
allow_missings=True,
)
# create validation set (predict=True) which means to predict the last max_prediction_length points in time
# for each series
validation = TimeSeriesDataSet.from_dataset(
training,
train_df[lambda x: x.time_idx > training_cutoff],
# predict = True,
stop_randomization=True)
test = TimeSeriesDataSet.from_dataset(
training,
test_df,
# predict = True,
stop_randomization=True)
# create dataloaders for model
batch_size = 32 # set this between 32 to 128
train_dataloader = training.to_dataloader(train=True, batch_size=batch_size, num_workers=0)
val_dataloader = validation.to_dataloader(train=False, batch_size=batch_size * 10, num_workers=0)
test_dataloader = test.to_dataloader(train=False, batch_size=batch_size * 10, num_workers=0)
##
#Train the Temporal Fusion Transformer
# ### Find optimal learning rate
pl.seed_everything(0)
trainer = pl.Trainer(
gpus=0,
# clipping gradients is a hyperparameter and important to prevent divergance
# of the gradient for recurrent neural networks
gradient_clip_val=0.1,
)
tft = TemporalFusionTransformer.from_dataset(
training,
# not meaningful for finding the learning rate but otherwise very important
learning_rate=0.03,
hidden_size=16, # most important hyperparameter apart from learning rate
# number of attention heads. Set to up to 4 for large datasets
attention_head_size=1,
dropout=0.1, # between 0.1 and 0.3 are good values
hidden_continuous_size=8, # set to <= hidden_size
output_size=7, # 7 quantiles by default
loss=QuantileLoss(),
# reduce learning rate if no improvement in validation loss after x epochs
reduce_on_plateau_patience=4,
)
print(f"Number of parameters in network: {tft.size()/1e3:.1f}k")
# ### Train model
# configure network and trainer
early_stop_callback = EarlyStopping(monitor="val_loss", min_delta=1e-4, patience=10, verbose=False, mode="min")
lr_logger = LearningRateMonitor() # log the learning rate
logger = TensorBoardLogger("lightning_logs") # logging results to a tensorboard
trainer = pl.Trainer(
max_epochs=30,
gpus=0,
weights_summary="top",
gradient_clip_val=0.1,
limit_train_batches=30, # coment in for training, running valiation every 30 batches
# fast_dev_run=True, # comment in to check that networkor dataset has no serious bugs
callbacks=[lr_logger, early_stop_callback],
logger=logger,
)
tft = TemporalFusionTransformer.from_dataset(
training,
learning_rate=0.001,
hidden_size=16,
attention_head_size=1,
dropout=0.1,
hidden_continuous_size=16,
output_size=7, # 7 quantiles by default
loss=QuantileLoss(),
log_interval=10, # uncomment for learning rate finder and otherwise, e.g. to 10 for logging every 10 batches
reduce_on_plateau_patience=4,
)
print(f"Number of parameters in network: {tft.size()/1e3:.1f}k")
##
# fit network
trainer.fit(
tft,
train_dataloader=train_dataloader,
val_dataloaders=val_dataloader,
)
##
# Specify a path
PATH = "../model/all_plants_2_7_entire_model.pt"
# Save
torch.save(trainer, PATH)
# Load
model = torch.load(PATH)
##
# load the best model according to the validation loss
# (given that we use early stopping, this is not necessarily the last epoch)
best_model_path = model.checkpoint_callback.best_model_path
best_tft = TemporalFusionTransformer.load_from_checkpoint(best_model_path)
# feature importants
prediction_tft_raw, pred_idx_df = best_tft.predict(test_dataloader, mode="raw", return_index = True)
interpretation = best_tft.interpret_output(prediction_tft_raw, reduction="sum")
figs_imp = best_tft.plot_interpretation(interpretation)
I don't understand why the following error produced by the raw predict .
prediction_tft_raw, pred_idx_df = best_tft.predict(test_dataloader, mode="raw", return_ ...: index = True) --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-92-4a2ba89f2e3c> in <module> ----> 1 prediction_tft_raw, pred_idx_df = best_tft.predict(test_dataloader, mode="raw", return_index = True) ~/.local/lib/python3.6/site-packages/pytorch_forecasting/models/base_model.py in predict(self, data, mode, return_index, return_decoder_lengths, batch_size, num_workers, fast_dev_run, show_progress_bar, return_x, mode_kwargs, **kwargs) 1058 output = _torch_cat_na(output) 1059 elif mode == "raw": -> 1060 output = _concatenate_output(output) 1061 1062 # generate output
~/.local/lib/python3.6/site-packages/pytorch_forecasting/models/base_model.py in _concatenate_output(output)
90 # concatenate simple tensors
91 if isinstance(v0, torch.Tensor):
---> 92 output_cat[name] = _torch_cat_na([out[name] for out in output])
93 # concatenate list of tensors
94 elif isinstance(v0, (tuple, list)) and len(v0) > 0:
~/.local/lib/python3.6/site-packages/pytorch_forecasting/models/base_model.py in _torch_cat_na(x)
68 for xi in x
69 ]
---> 70 return torch.cat(x, dim=0)
71
72
RuntimeError: Sizes of tensors must match except in dimension 0. Got 98 and 97 in dimension 3 (The offending index is 5)
no, and would be interested in hearing any progress...
On Mon, Nov 8, 2021 at 5:56 AM Adam @.***> wrote:
@zhangxin0611 https://github.com/zhangxin0611 I met the same error, have you solved it?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jdb78/pytorch-forecasting/issues/565#issuecomment-962788345, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALI77B2HY5Y7W3C5JT7STJLUK5C7DANCNFSM46WGONOA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
Do you have a reproducible example with data? I just cannot reproduce the error.
I'm working on this project again - my notebooks can be reached through the comment here as well: https://github.com/jdb78/pytorch-forecasting/issues/579
Code to reproduce the problem
Here is a link to a colab notebook to run the code and reproduce the output: https://github.com/Bassie1/notebooks/blob/main/model_github.ipynb And here is a link to the dataset used: https://github.com/Bassie1/notebooks/blob/main/github_data.csv
On Wed, Dec 1, 2021 at 4:46 PM Jan Beitner @.***> wrote:
Do you have a reproducible example with data? I just cannot reproduce the error.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jdb78/pytorch-forecasting/issues/565#issuecomment-983711716, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALI77BZAZMP34COFSSY3CI3UOYYKXANCNFSM46WGONOA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.