tsugumi-sys
The problem of changing data shape.
Hello~ I have a question on different data shape to train convlstm model.
If the original data shape is [20, 10000, 64, 64], with the first 10 frames as input and the last 10 frames as predictions, and I want to change it so that the first 15 frames are input and the last 5 frames are predictions, how should I modify the code?
code: def main(): ### # Common Params ### artifact_dir = "./test_label_frame2" input_seq_length = 15 train_batch_size = 32 validation_bath_size = 16 ### # Setup Pipeline ### model_params: Seq2SeqParams = { "input_seq_length": input_seq_length, "num_layers": 2, "num_kernels": 64, "return_sequences": False, "convlstm_params": { "in_channels": 1, "out_channels": 1, "kernel_size": (3, 3), "padding": "same", "activation": "relu", "frame_size": (64, 64), "weights_initializer": WeightsInitializer.He, }, } erorr: Traceback (most recent call last):
File ~/anaconda3/envs/convlstm/lib/python3.11/site-packages/spyder_kernels/py3compat.py:356 in compat_exec exec(code, globals, locals)
File /data/ConvLSTM-test/examples/moving_mnist_convlstm.py:72 main()
File /data/ConvLSTM-test/examples/moving_mnist_convlstm.py:68 in main experimenter.run()
File /data/ConvLSTM-test/pipelines/experimenter.py:44 in run self.__evaluate()
File /data/ConvLSTM-test/pipelines/experimenter.py:74 in __evaluate evaluator.run()
File /data/ConvLSTM-test/pipelines/evaluator.py:34 in run pred_frames = self.__predict_frames(input, label) # 對輸入預測
File /data/ConvLSTM-test/pipelines/evaluator.py:63 in __predict_frames pred_frames[:, :, frame_idx] = self.model(
IndexError: index 5 is out of bounds for dimension 2 with size 5
@houjanet
I need to add label_seq_length parameter and change models code.
Please wait a few days, I'll fix it:)
@houjanet
Now, you can use label_seq_length for Seq2Seq model (ConvLSTM). Please try it :)
I'll also support for the other models.
https://github.com/tsugumi-sys/SA-ConvLSTM-Pytorch/blob/fd8a4601dce5b302f5e86271253a73039a3097c4/convlstm/seq2seq.py#L15
error:
Traceback (most recent call last):
File ~/anaconda3/envs/convlstm/lib/python3.11/site-packages/spyder_kernels/py3compat.py:356 in compat_exec exec(code, globals, locals)
File /data/SA-ConvLSTM-Pytorch-main/examples/moving_mnist_convlstm.py:71 main()
File /data/SA-ConvLSTM-Pytorch-main/examples/moving_mnist_convlstm.py:67 in main experimenter.run()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/experimenter.py:43 in run self.__evaluate()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/experimenter.py:70 in __evaluate evaluator.run()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/evaluator.py:33 in run pred_frames = self.__predict_frames(input, label)
File /data/SA-ConvLSTM-Pytorch-main/pipelines/evaluator.py:60 in __predict_frames pred_frames[:, :, frame_idx] = self.model(
IndexError: index 5 is out of bounds for dimension 2 with size 5
@houjanet Thanks for your error report. Could you share the whole code so that I can reproduce the same error?
moving_mnist_convlstm.py code:
import os
os.chdir('/data/SA-ConvLSTM-Pytorch-main')
print(os.getcwd())
from torch import nn
from torch.optim import Adam
from convlstm.seq2seq import Seq2Seq, Seq2SeqParams
from core.constants import WeightsInitializer
from data_loaders.moving_mnist import MovingMNISTDataLoaders
from pipelines.experimenter import Experimenter
from pipelines.trainer import TrainingParams
from pipelines.utils.early_stopping import EarlyStopping
def main():
###
# Common Params
###
artifact_dir = "./test"
**input_seq_length = 15**
train_batch_size = 32
validation_bath_size = 16
###
# Setup Pipeline
###
model_params: Seq2SeqParams = {
"input_seq_length": input_seq_length,
"num_layers": 2,
"num_kernels": 64,
"return_sequences": False,
"convlstm_params": {
"in_channels": 1,
"out_channels": 1,
"kernel_size": (3, 3),
"padding": "same",
"activation": "relu",
"frame_size": (64, 64),
"weights_initializer": WeightsInitializer.He,
},
}
model = Seq2Seq(**model_params)
training_params: TrainingParams = {
"epochs": 1,
"loss_criterion": nn.BCELoss(reduction="sum"),
"accuracy_criterion": nn.L1Loss(),
"optimizer": Adam(model.parameters(), lr=1e-4),
"early_stopping": EarlyStopping(
patience=30,
verbose=True,
delta=0.0001,
),
"metrics_filename": "metrics.csv",
}
print("Loading dataset ...")
data_loaders = MovingMNISTDataLoaders(
train_batch_size=train_batch_size,
validation_batch_size=validation_bath_size,
input_frames=model_params["input_seq_length"],
label_frames=1,
split_ratios=[0.7, 0.299, 0.001],
)
experimenter = Experimenter(artifact_dir, data_loaders, model, training_params)
experimenter.run()
if __name__ == "__main__":
main()
I only changed this part: input_seq_length = 15, and the error would be:
Traceback (most recent call last):
File ~/anaconda3/envs/convlstm/lib/python3.11/site-packages/spyder_kernels/py3compat.py:356 in compat_exec
exec(code, globals, locals)
File /data/SA-ConvLSTM-Pytorch-main/examples/moving_mnist_convlstm.py:72
main()
File /data/SA-ConvLSTM-Pytorch-main/examples/moving_mnist_convlstm.py:68 in main
experimenter.run()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/experimenter.py:43 in run
self.__evaluate()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/experimenter.py:70 in __evaluate
evaluator.run()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/evaluator.py:33 in run
pred_frames = self.__predict_frames(input, label)
File /data/SA-ConvLSTM-Pytorch-main/pipelines/evaluator.py:60 in __predict_frames
pred_frames[:, :, frame_idx] = self.model(
IndexError: index 5 is out of bounds for dimension 2 with size 5
@houjanet
Thanks! You can try this? I added label_seq_lengh and modified the label_frames argument of the MovingMNIST data loader.
input_seq_length = 15
label_seq_length = 5
model_params: Seq2SeqParams = {
"input_seq_length": input_seq_length,
"label_seq_length": label_seq_length,
"num_layers": 2,
"num_kernels": 64,
"return_sequences": False,
"convlstm_params": {
"in_channels": 1,
"out_channels": 1,
"kernel_size": (3, 3),
"padding": "same",
"activation": "relu",
"frame_size": (64, 64),
"weights_initializer": WeightsInitializer.He,
},
}
model = Seq2Seq(**model_params)
training_params: TrainingParams = {
"epochs": 1,
"loss_criterion": nn.BCELoss(reduction="sum"),
"accuracy_criterion": nn.L1Loss(),
"optimizer": Adam(model.parameters(), lr=1e-4),
"early_stopping": EarlyStopping(
patience=30,
verbose=True,
delta=0.0001,
),
"metrics_filename": "metrics.csv",
}
print("Loading dataset ...")
data_loaders = MovingMNISTDataLoaders(
train_batch_size=train_batch_size,
validation_batch_size=validation_bath_size,
input_frames=model_params["input_seq_length"],
label_frames=model_params["input_seq_length"],
split_ratios=[0.7, 0.299, 0.001],
)
I tried this code, and the error changed to:
Traceback (most recent call last):
File ~/anaconda3/envs/convlstm/lib/python3.11/site-packages/spyder_kernels/py3compat.py:356 in compat_exec
exec(code, globals, locals)
File /data/SA-ConvLSTM-Pytorch-main/examples/moving_mnist_convlstm.py:71
main()
File /data/SA-ConvLSTM-Pytorch-main/examples/moving_mnist_convlstm.py:67 in main
experimenter.run()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/experimenter.py:43 in run
self.__evaluate()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/experimenter.py:70 in __evaluate
evaluator.run()
File /data/SA-ConvLSTM-Pytorch-main/pipelines/evaluator.py:33 in run
pred_frames = self.__predict_frames(input, label)
File /data/SA-ConvLSTM-Pytorch-main/pipelines/evaluator.py:58 in __predict_frames
pred_frames[:, :, frame_idx] = self.model(input)
RuntimeError: The expanded size of the tensor (1) must match the existing size (5) at non-singleton dimension 1. Target sizes: [1, 1, 64, 64]. Tensor sizes: [5, 64, 64]
@houjanet hmm, are you using your custom dataset? If so, I want to know more detail code about it
No, I just used the original moving_mnist dataset to try input_seq_length = 15 and label_seq_length = 5