mmtracking
mmtracking copied to clipboard
Maybe a small bug about test progress bar in multi_gpu_test().
During distributed training, sot model use multi_gpu_test() func to test all videos on several GPUs. But the progress bar is updated on rank 0.
https://github.com/open-mmlab/mmtracking/blob/e79491ec8f0b8c86fda947fbaaa824c66ab2a991/mmtrack/apis/test.py#L160C34-L160C34
The number of frames on each rank is not equal, in my run, the number of frames on rank0 is less than total_frames / world_size, so the progress bar will be less than the actual total number of frames. For example, I used 3 gpu, and total 21007 frames(got10k val), and here is some out info. the final progress only 5565*3=16695.
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ] 16695/21007, 99.7 task/s, elapsed: 167s, ETA: 43s rank 0 world_size 3 cnt 5565. rank 1 world_size 3 cnt 7072. rank 2 world_size 3 cnt 8370.
Hello @MisakaMikt Haven't you got this MMCV compatibility error?