nuscenes-devkit icon indicating copy to clipboard operation
nuscenes-devkit copied to clipboard

Feature request: Add parallel metric calculation.

Open ternaus opened this issue 5 years ago • 1 comments

At the moment mAP calculation is slow. We can calculate values for different thresholds and classes in parallel.

ternaus avatar Oct 12 '19 15:10 ternaus

Hey @ternaus, I'd love to work on this.

Here's how I plan to implement this:

  1. Parallelly compute the per class mAPs for each IoU threshold and save them in a temp file. (temp files are necessary because parallel processes can't use global variables of the master process, I'm all in for ideas, let me know if you there's a better way to do this.)
  2. Once all parallel processes are finished, read the temp files and calculate and print the overall mAP and remove the temp files.
  3. We can also save all the class-wise computed APs for each threshold in a JSON file, something like this

Let me know your thoughts.

pyaf avatar Oct 13 '19 07:10 pyaf