So confused......a problem about different evaluation precision results , with one model.

[2401ch]

Hello, I evaluate the model accuracy to get cmc and mAP through several different codes, ftResnet50 model provided by this project.

1. evaluate_gpu.py, from this project, Person_reID_baseline_pytorch. Rank@1:0.888361 Rank@5:0.952197 Rank@10:0.969715 mAP:0.715947

2. eval_reid.py, from reid-strong-baseline, https://github.com/michuanhaohao/reid-strong-baseline directory : ./reid-strong-baselinedata/datasets mAP: 67.6% CMC curve Rank-1 : 81.1% Rank-5 : 94.4% Rank-10 : 96.5% Rank-20 : 97.9%

3. rank_cylib, from deep-person-reid, https://github.com/KaiyangZhou/deep-person-reid directory: ./deep-person-reid/torchreid/metrics mAP: 67.6% CMC curve Rank-1 : 81.1% Rank-5 : 94.4% Rank-10 : 96.5% Rank-20 : 97.9% It is same with 2.

Why the precision results are not same???

[layumi]

@chenghan1995 Sorry. I did not check other people's open-source codes. But I am very sure that my code is aligned with the Matlab code in the original Market-1501 paper. You may notice that I also provide mat file. You could use it to evaluate in the original Matlab evaluation code. You will get the same result.

[tangsipeng]

@chenghan1995 found the same problem train on reid-strong-baseline myself and get The test feature is normalized 2020-01-06 16:12:57,226 reid_baseline.inference INFO: Validation Results 2020-01-06 16:12:57,226 reid_baseline.inference INFO: mAP: 85.7% 2020-01-06 16:12:57,226 reid_baseline.inference INFO: CMC curve, Rank-1 :94.3% 2020-01-06 16:12:57,226 reid_baseline.inference INFO: CMC curve, Rank-5 :98.3% 2020-01-06 16:12:57,226 reid_baseline.inference INFO: CMC curve, Rank-10 :98.9%

but use this model on evaluate_gpu.py Rank@1:0.403800 Rank@5:0.743171 Rank@10:0.880344 mAP:0.276178

[layumi]

Hi @tangsipeng I am not sure the reason caused by other evaluation code.

One common mistake of evaluation code might be whether you remove the image from the same camera. In the original code of Market-1501, we neglect the image of the same person from the same camera (since they are too simple). We only intend to find the cross-camera images of the same person.

The python code is at this line: https://github.com/layumi/Person_reID_baseline_pytorch/blob/master/evaluate_gpu.py#L25