agethen
Some question about Sequence markers.
Thank you for sharing your work, and I have some questiones about the sequence markers.
According to build_hdf5.py, the "sequence" should has the shape T*N, where T =20 and N =16. But in the prototxt, "batch_size: 19 # T + (T-1)", so why is not 20?
And after "slice_seq" layer, the "seq_enc" has the shape 10*16, which the first row is 0 and the other is 1, and the "seq_dec" is all 1, right? So what is the shape of "seq_dec"?
When testing, what the squence markers should be?
Any suggestions would be appreciated. Thanks!
The reason for 19 timesteps being used only in that example is from the original ConvLSTM. If you check their Theano code, around line 140-150 in mnist_sequence_forecasting...., they take the last output of the encoding phase, concatenate it with 9 outputs of the decoding phase, and generate the predictions from that. That is, 10 steps during encoding, and 9 during decoding. I also saw that later only, and just never adjusted the input sequence, so that explains why it still says 20 there!
You could of course change that behavior! In the example I try to stay close to the Theano model published with the paper.
Your understanding of seq_enc is correct. seq_dec is simply all ones (as we are copying the cell and hidden state from the encoding phase, and not reseting it). The markers are constant, and don't change for the test phase.
On further inspection, I realize there might be a problem with the way the sequence markers are loaded in the example.
If the sequence file contains a tensor 20 x 16, and we specify batch_size: 19, the 20th time-step would now become the first time-step in the next batch. The second time-step would then contain the zero, which is incorrect behavior.
As we still need only T=19, I need to either update seq.h5, or split the Caffe blob such that the last row is discarded.
Thanks for pointing this out!