KittiBox icon indicating copy to clipboard operation
KittiBox copied to clipboard

Reproduce result for Kitti benchmark

Open nghiattran opened this issue 7 years ago • 4 comments

@MarvinTeichmann How reproduce results in your Multinet paper? I tried to add resnet decoders and compare them with the VGG decoder but when I submit results onto http://www.cvlibs.net/datasets/kitti/, it turns out that VGG can only reach 50% on moderate. I used min_conf=0.50, tau=0.35 and exactly same setting as in provided pretrained VGG.

nghiattran avatar May 11 '17 14:05 nghiattran

Strange. What is your validation score?

MarvinTeichmann avatar May 13 '17 11:05 MarvinTeichmann

Score produced by tv-analyze:

train easy train moderate train hard val easy val moderate val hard speed (msec) speed (fps) post (msec)
VGG 93.25% 84.10% 69.09% 94.27% 86.32% 70.78% 104.4554 9.5735 3.5804

I don't have a screenshot of Kitti result because I submitted more than a week ago and it has been deleted.

nghiattran avatar May 13 '17 15:05 nghiattran

I also submitted a result using the model (KittiBox_pretrained.zip), which is provided by author (MarvinTeichmann) and got the 45.22% for the moderate in a car detection task. (Easy: 71.99% and Hard: 36.83%) . Have you produced results using the Kitti test dataset?

robin2002 avatar May 31 '17 02:05 robin2002

This is the result for the test set. This result is higher than what I originally got but it is very low compared to what is in the official paper.

screenshot from 2017-06-01 00-09-32

nghiattran avatar Jun 01 '17 04:06 nghiattran