DenseTNT icon indicating copy to clipboard operation
DenseTNT copied to clipboard

eval_batch_size

Open Guozhongyuan opened this issue 4 years ago • 5 comments

When use a small eval_batch_size, the eval results will be bad, because global_graph() use the max length in a batch to pad zero in utils.merge_tensors(). Change this 'merge_tensors' to use a fixed length, and then use different eval_batch_size will get the same eval result. lADPGR5pkZNRijDNAQbNBDg_1080_262

Guozhongyuan avatar Nov 23 '21 06:11 Guozhongyuan

Hello! Can you reach the best results show in the readme.md? @Guozhongyuan

HMTJYQS avatar Dec 27 '21 02:12 HMTJYQS

Hello! Can you reach the best results show in the readme.md? @Guozhongyuan

Close to, but worse

Guozhongyuan avatar Dec 28 '21 03:12 Guozhongyuan

Can you list your results here? The results in readme.md are averaged. @Guozhongyuan

GentleSmile avatar Dec 28 '21 03:12 GentleSmile

Can you list your results here? The results in readme.md are averaged. @Guozhongyuan

  minADE minFDE MR
Optimization, pad length 250 0.9015 1.3073 0.0774
Optimization, not pad 0.8373 1.3294 0.0788
NMS, pad length 250 0.9082 1.3353 0.0896
NMS, not pad 0.8464 1.3647 0.0922
Set predictor, not pad 0.8135 1.2782 0.0848

Guozhongyuan avatar Dec 28 '21 08:12 Guozhongyuan

This result is far away from our expected result. The MR should be from 0.069 to 0.071. What about the result of using the default training command and evaluation command without changing any code?

GentleSmile avatar Dec 28 '21 09:12 GentleSmile