Video-Classification-2-Stream-CNN
Video-Classification-2-Stream-CNN copied to clipboard
wadhwasahil
did you upload the lstm related code? and can you share the experiment result?
We couldn't try out the LSTMs due to lack of enough GPU memory. If you wish to implement it, just follow the 2nd paper from the README. You need to compute the frame-wise feature vector from the FC layer of each video and feed it to a distinct timestep of an LSTM.
As for the results, I don't have a report, but the 2-stream CNNs yielded accuracy a bit lower than those given in section 4.2 of the 2nd paper from the README. Spatial - 65%-70% Temporal - 50%-55%
the main consume comes from the feature extraction pre-frame, did optical-flow helps save computation?
any idea about how to reduce consume in the front-end feature extraction phase?
thks!
ren
2016-07-14 15:54 GMT+08:00 Ashar Javed [email protected]:
We couldn't try out the LSTMs due to lack of enough GPU memory. If you wish to implement it, just follow the 2nd paper from the README. You need to compute the frame-wise feature vector from the FC layer of each video and feed it to a distinct timestep of an LSTM.
As for the results, I don't have a report, but the 2-stream CNNs yielded accuracy a bit lower than those given in section 4.2 of the 2nd paper from the README. Spatial - 65%-70% Temporal - 50%-55%
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wadhwasahil/Video-Classification-2-Stream-CNN/issues/3#issuecomment-232590004, or mute the thread https://github.com/notifications/unsubscribe/ADTRYS-ukVrSk8-1-KP6mAttQa9mrk3dks5qVesvgaJpZM4JMKKI .
@stillbreeze I know we need to compute the frame-wise feature vector from the FC layer of each video and feed it to a distinct timestep of an LSTM,but how can we use keras to make it ? the Sequential model can't work? I'm confused how to stack the outputs of the CNN layers to the LSTM,can you share some idea?
@wadhwasahil
how many epochs did it take for the accuracy value to settle(for spatial stream) ? and how long did it take on your system. Please let me know the specifications.
cheers.
@kvignesh1420 In the code we used 50 epochs for the spatial stream, but I am pretty sure, we got the results before that. @stillbreeze Do you know the specs for temporal training?
@guoyang007 : Sorry for the late reply. One straightforward way which most papers also follow is to not train the whole network end-to-end. So, you train the CNN, freeze the weights and then obtain the FC vectors and then feed them into a separate LSTM model.
@wadhwasahil : Sorry, but can't remember the training configuration.
@wadhwasahil @guoyang007
can you please upload the lstm code if available. I am not able to extract the FC feature vectors and feed it into LSTM. Help me out.
But still @kvignesh1420 , how did u solve it? It would be better for other people if you could share your views or code.
@wadhwasahil give me some time. I will upload the whole procedure.. :D