deep_sort_pytorch icon indicating copy to clipboard operation
deep_sort_pytorch copied to clipboard

what's the problem when i test on MOT16 dataset?

Open 394781865 opened this issue 5 years ago • 5 comments

2020-04-26 10:19:16 [INFO]: Evaluate seq: MOT16-05 IDF1 IDP IDR Rcll Prcn GT MT PT ML FP FN IDs FM MOTA MOTP IDt IDa IDm MOT16-05 0.0% 0.0% nan% nan% 0.0% 0 0 0 0 1747 0 0 0 -inf% nan 0 0 0 OVERALL 0.0% 0.0% nan% nan% 0.0% 0 0 0 0 1747 0 0 0 -inf% nan 0 0 0

394781865 avatar Apr 26 '20 02:04 394781865

2020-04-26 10:19:16 [INFO]: Evaluate seq: MOT16-05 IDF1 IDP IDR Rcll Prcn GT MT PT ML FP FN IDs FM MOTA MOTP IDt IDa IDm MOT16-05 0.0% 0.0% nan% nan% 0.0% 0 0 0 0 1747 0 0 0 -inf% nan 0 0 0 OVERALL 0.0% 0.0% nan% nan% 0.0% 0 0 0 0 1747 0 0 0 -inf% nan 0 0 0

Check if the data being read from the results files is actually read. In my case, I had to modify the eval script a little to work.

Nishzzz avatar Apr 28 '20 06:04 Nishzzz

@Nishzzz can you clarify the code that you modified? thanks

vantienpham avatar May 02 '20 09:05 vantienpham

2020-04-26 10:19:16 [INFO]: Evaluate seq: MOT16-05 IDF1 IDP IDR Rcll Prcn GT MT PT ML FP FN IDs FM MOTA MOTP IDt IDa IDm MOT16-05 0.0% 0.0% nan% nan% 0.0% 0 0 0 0 1747 0 0 0 -inf% nan 0 0 0 OVERALL 0.0% 0.0% nan% nan% 0.0% 0 0 0 0 1747 0 0 0 -inf% nan 0 0 0

Check if the data being read from the results files is actually read. In my case, I had to modify the eval script a little to work.

请问你怎么改的eval script?

Angel0003 avatar Jun 10 '20 08:06 Angel0003

Hi everyone, I managed to figure out how to change the code in yolov3_deepsort_eval.py such that it will run correctly. I realized that the documentation on how to do this is a bit lacking, so I'll chip in on how this can be done.

nowadays when you download the MOT16 data file from here, the "video" actually comes as a sequence of frames stored in MOT16/train/MOT16-XX/img1. You need to convert this into a video file for the tracker to track items in it. The resulting video (converted from frames in MOT16/train/MOT16-XX/img1) is the video that is being referred to on line 34 in yolov3_deepsort_eval.py (video_path = data_root+"/"+seq+"/video/video.mp4"). Change video_path to the path where your video is stored.

This is probably obvious, but on line 81 (data_root = 'data/dataset/MOT16/train/') data_root needs to refer to the root directory of the folder that you download from here.

This is probably obvious too, but for some reason, on line 36 (with VideoTracker(cfg, args, video_path, result_filename) as vdo_trk:) for some reason there are 4 arguments. This is incorrect, since the VideoTracker class in yolov3_deepsort.py only takes 3 arguments. The extra (incorrect) argument in this case is the last one, result_filename. Remove it so that the resulting line is "with VideoTracker(cfg, args, video_path) as vdo_trk:"

This is where things get a little less obvious. Two files are generated as a result of running the VideoTracker (from yolov3_deepsort.py) for an input video. These two files (results.avi and results.txt) are stored in the path that is provided in the argument "--save_path" (line 65). On line 65 (parser.add_argument("--save_path", type=str, default="./demo/demo.avi")) change default="./demo/demo.avi" to whichever path you want results.avi and result.txt to be stored. Note that if this is not changed, by default only demo.avi (which is results.avi in yolov3_deepsort.py) will be generated. If a path (as opposed to path to a specific file) is specified, then VideoTracker will generate and store the results of tracking as results.avi and results.txt, by default, as specified in line 59, 60 in yolov3_deepsort.py (59: self.save_video_path = os.path.join(self.args.save_path, "results.avi") 60: self.save_results_path = os.path.join(self.args.save_path, "results.txt")). Another work-around, of course, would be to specify a path as a flag when calling the function from the command line (i.e., python yolov3_deepsort_eval.py --save_path path/you/want/to/store/results/in).

Lastly, looking into line 33 (result_filename = os.path.join(result_root, '{}.txt'.format(seq))) one will notice that result_filename is actually meant to refer to the results.txt file that I mentioned above in (4). i.e., the results.txt file that is generated as a result of running the VideoTracker. You have to modify result_filename such that it refers to the path at which results.txt is stored for a given video. For instance, if --save_path is "./output/", then result_filename="./output/results.txt"

One major issue I noticed is that, even if the user passes in a path that does not lead to any legitimate results.txt file, the program does not give any indication (error/warning), that the user is running the program incorrectly. Instead, the mistake is reflected as a bunch of 0%s and nan's in the summary printed at the end of the program.

I hope this helps! If I'm missing anything, do leave a comment below!

alexrider1105 avatar Aug 15 '20 18:08 alexrider1105

Hi everyone, I managed to figure out how to change the code in yolov3_deepsort_eval.py such that it will run correctly. I realized that the documentation on how to do this is a bit lacking, so I'll chip in on how this can be done.

nowadays when you download the MOT16 data file from here, the "video" actually comes as a sequence of frames stored in MOT16/train/MOT16-XX/img1. You need to convert this into a video file for the tracker to track items in it. The resulting video (converted from frames in MOT16/train/MOT16-XX/img1) is the video that is being referred to on line 34 in yolov3_deepsort_eval.py (video_path = data_root+"/"+seq+"/video/video.mp4"). Change video_path to the path where your video is stored.

This is probably obvious, but on line 81 (data_root = 'data/dataset/MOT16/train/') data_root needs to refer to the root directory of the folder that you download from here.

This is probably obvious too, but for some reason, on line 36 (with VideoTracker(cfg, args, video_path, result_filename) as vdo_trk:) for some reason there are 4 arguments. This is incorrect, since the VideoTracker class in yolov3_deepsort.py only takes 3 arguments. The extra (incorrect) argument in this case is the last one, result_filename. Remove it so that the resulting line is "with VideoTracker(cfg, args, video_path) as vdo_trk:"

This is where things get a little less obvious. Two files are generated as a result of running the VideoTracker (from yolov3_deepsort.py) for an input video. These two files (results.avi and results.txt) are stored in the path that is provided in the argument "--save_path" (line 65). On line 65 (parser.add_argument("--save_path", type=str, default="./demo/demo.avi")) change default="./demo/demo.avi" to whichever path you want results.avi and result.txt to be stored. Note that if this is not changed, by default only demo.avi (which is results.avi in yolov3_deepsort.py) will be generated. If a path (as opposed to path to a specific file) is specified, then VideoTracker will generate and store the results of tracking as results.avi and results.txt, by default, as specified in line 59, 60 in yolov3_deepsort.py (59: self.save_video_path = os.path.join(self.args.save_path, "results.avi") 60: self.save_results_path = os.path.join(self.args.save_path, "results.txt")). Another work-around, of course, would be to specify a path as a flag when calling the function from the command line (i.e., python yolov3_deepsort_eval.py --save_path path/you/want/to/store/results/in).

Lastly, looking into line 33 (result_filename = os.path.join(result_root, '{}.txt'.format(seq))) one will notice that result_filename is actually meant to refer to the results.txt file that I mentioned above in (4). i.e., the results.txt file that is generated as a result of running the VideoTracker. You have to modify result_filename such that it refers to the path at which results.txt is stored for a given video. For instance, if --save_path is "./output/", then result_filename="./output/results.txt"

One major issue I noticed is that, even if the user passes in a path that does not lead to any legitimate results.txt file, the program does not give any indication (error/warning), that the user is running the program incorrectly. Instead, the mistake is reflected as a bunch of 0%s and nan's in the summary printed at the end of the program.

I hope this helps! If I'm missing anything, do leave a comment below!

would you please share the modified code to conduct evaluation on multiple sequences. it will be a great help

ssbilakeri avatar Jul 12 '21 05:07 ssbilakeri