temporal-shift-module
temporal-shift-module copied to clipboard
mit-han-lab
Training script for online TSM
Hi! Thanks for the impressive work with publicly accessible source code here :) I am trying to train another application for online TSM with different datasets and adding a few adjustments. The repo currently only has the offline version of training script. Is it possible for you also providing the training script for the online TSM? Thank you very much!
Hi, for online recognition, we need to use a uni-directional TSM, but we have not included the code. But it would be easy to modify by yourself (just change the activation shift part to shift in one side). The other training is just the same as the offline part.
After training, you may need to unroll the model to take one image as input. Now I do it manually, but I will try to provide an automated script if i have time (it would still be easy though).
hi， Does the training of uni-directional TSM needs the mutli rgb frames as input, and use the segemental consensus to deal with it? Or use a single frame as input of model？
For uni-directional TSM, we used multiple frames as input during training, while only one frame as input during testing.
@tonylins Hi Author, Thanks for the impressive work with publicly accessible source code here. I am trying to train another application for online TSM with different datasets. Could you provide the training script of Online model and how to modify the code to Online uni-directional TSM? Looking forward your reply! Thank you very much!
Hi, Firstly, thank you for this repo, it's really an impressive work. I have a request related to the subject of this issue so I will post it here. I was trying to tune the online model downloaded in the online demo code. I saw that some changes needed to be made in order to make online training work, but I don't really see where and how. So if I can get a hint on what to modify I'd appreciate that. Thanks in advance!
Hi! Thanks for the impressive work with publicly accessible source code here :) I am trying to train another application for online TSM with different datasets and adding a few adjustments. The repo currently only has the offline version of training script. Is it possible for you also providing the training script for the online TSM? Thank you very much!
Hi have u solved it?
@Nauman007 I do. But I observed significant accuracy drop. I took the mobilenetv2 implementation from the online-demo/ folder and made a new training script with the same parameters as the offline training. I observed a 15% accuracy drop and very poor demo results. I suspect the number of segment to be very important for good accuracy (I used only 1 segment)
For uni-directional TSM, we used multiple frames as input during training, while only one frame as input during testing.
Hi, should the sampling strategy of training be consistent with testing, when training the online model? Because the online model considers the continuous frames as the input clip when processing the frame stream.
Hi, for online recognition, we need to use a uni-directional TSM, but we have not included the code. But it would be easy to modify by yourself (just change the activation shift part to shift in one side). The other training is just the same as the offline part.
After training, you may need to unroll the model to take one image as input. Now I do it manually, but I will try to provide an automated script if i have time (it would still be easy though).
Hi，thanks your reply. The online training method is equal to 'online_demo/main.py' where has shift buffer?
