pytorch-vsumm-reinforce
pytorch-vsumm-reinforce copied to clipboard
KaiyangZhou
How can get change points using KTS?
I tried to get change points using KTS code. But i couldn't get proper change points.
If someone get change points using KTS, please help me?
@SinDongHwan http://lear.inrialpes.fr/people/potapov/med_summaries.php are you use this code? can you reach the performace that the paper provided. the reward hardly rises.
@zouying-sjtu yes, i used. i edited something (https://github.com/SinDongHwan/pytorch-vsumm-reinforce/blob/master/utils/generate_dataset.py) but i can't reach. change points of our and change points of dataset are differ. i will test. how result.
I tried KTS on SumMe, taking googlenet features as input as the paper stated. First I don't know how to decide the number of change points (argument m in the cpd_auto function). Even if I set the same m as the the author did, the segmenting change points are totally different. I tried KTS on both full frames and downsampled video, but still can reproduce the results in the *.h5 files.
Another thing confused me is that the fps of original videos are different, some of them ( for example cooking) are 15 fps. But all of them are treated as 30 fps and downsampled to 2 fps ( 15 times shorter). I am not sure if there would be some problems.
I've faced to have different change points. And I've had bad result when i used googlenet to extract features. So, i tried to extract features using resnet101. this was a good result. but change points was different.
I've not solved about change points difference.
in some papers, author said "use 1 or 2fps". so i think if video has 15fps, 1fps downsampling. This is just my think.
I tried KTS on SumMe, taking googlenet features as input as the paper stated. First I don't know how to decide the number of change points (argument m in the cpd_auto function). Even if I set the same m as the the author did, the segmenting change points are totally different. I tried KTS on both full frames and downsampled video, but still can reproduce the results in the *.h5 files.
Another thing confused me is that the fps of original videos are different, some of them ( for example cooking) are 15 fps. But all of them are treated as 30 fps and downsampled to 2 fps ( 15 times shorter). I am not sure if there would be some problems.
@SinDongHwan Thanks for your reply! 1.
So, i tried to extract features using resnet101. this was a good result.
How do you know resnet101 features generate good results?
I tried both googlenet and resnet101, using all the frames/2fps downsampled frames, but all the results are just bad.
I tried to figure out how to properly set maximum number of change points (npc in function cpd_auto()), because the algorithm should automatically compute the best number of chang points for me but it just didn't work. So I did an experiment on video Air_Force_One of summe, downsampled to 2fps, so there are 300 feature vectors. Then I set argument ncp=300, the resulting number of change points is 275, which is obviously wrong, beacuse there are only 30 change points in the .h5 file provided by the author of this code repo.
The original videos are not just 30 fps or 15 fps, some are 25 fps. Air_Force_One and Statue of Liberty in summe are 25 fps. Still, I don't know it matters or not.
@JudeLiu Hello ,mate. have you solve the problem? I read the KTS paper and the code, but I still don't know how to set the maximum number of change points(nps). Looking forward to your reply,thx.
You can refer ”Category-specific video summarization" by Danila Ptapov,Matthijs Douze, Zaid Harchaouni,Cordelia Schmid. You can also refer https://github.com/TorRient/Video-Summarization-Pytorch and my repository( https://github.com/anaghazachariah/video_summary_generaton )
