2016_person_re-ID icon indicating copy to clipboard operation
2016_person_re-ID copied to clipboard

about the cuhk03 evaluation

Open zgplvyou opened this issue 7 years ago • 15 comments

thanks for your evaluation code , but i am a little confused about one line of it. could you tell me what 'ff = ff.ff1' and 'ff =ff.ff2' mean. i don't find where the 'ff , ff1' is.

zgplvyou avatar Dec 05 '17 01:12 zgplvyou

and why '[~,index]= sort(score,'ascend')' in market1501 evaluation and '[~,index]= sort(score,'descend')' in cuhk03 evaluation?

zgplvyou avatar Dec 05 '17 01:12 zgplvyou

Hi @zgplvyou ff1 is query feature ff2 is gallery feature We evaluate the result by the matching score. So we use sort

layumi avatar Dec 05 '17 02:12 layumi

i have used your market1501 evaluation code , so i know the 'sort' mean. and why sort by ‘ascend' in market1501 but 'descend' in cuhk03

zgplvyou avatar Dec 05 '17 02:12 zgplvyou

In market1501, I followed the original code using (x-y)^2 In cuhk, I rewrite the code using x*y (x,y have been normalized) since x^2 =1, y^2=1

layumi avatar Dec 05 '17 02:12 layumi

thanks for Zheng ge's answers!

zgplvyou avatar Dec 05 '17 02:12 zgplvyou

Sorry for one more question  about 'single-shot' and 'multi-shot'.
i know the 'single-query' and 'multi-query' due to your explanation. and i found the definition of 'single-shot' and 'multi-shot' that In general, when only an image pair is matched, the method is considered as single-shot recognition method. If matching is conducted between two sets of images, e.g. frames obtained from two separate trajectories. 

the test images of CUHK03 is obtained from 6 cameras(3 camera pairs[camera_1,camera_1'; camera_2,camera_2'; camera_3,camera_3']). the amount of test images is about 100 * 10.
so how to choose the query images and gallery images for single-shot and multi-shot. is the prob=[all test images in camera_1, camera_2, camera_3], gallery = [random choose one image of each identity in camera_1', camera_2', camera_3'] in single-shot right? but in this case, the matching is obtained from different cameras. i am confused of the choice of probe and gallery on single-shot and multi-shot.

zgplvyou avatar Dec 05 '17 04:12 zgplvyou

The original setting is single shot. Every time we select 1 query and 100 images from 100 person (another camera . For every person, we randomly selects one image) as gallery. And run 20 times to report the mean precision.

Multi-shot is similar to market1501. We select 1 query and all images (another camera) as gallery. Run one time.

layumi avatar Dec 05 '17 04:12 layumi

Thank you again! i have got the rank1, rank5 and map.

zgplvyou avatar Dec 05 '17 14:12 zgplvyou

thanks for your evaluation code , but i am a little confused about one line of it. could you tell me what 'load('cuhk03_feature_res_split6.mat')' mean. i don't find where the 'cuhk03_feature_res_split6.mat' is.

daichenchao avatar May 31 '18 13:05 daichenchao

Hi @daichenchao You need to generate it. I wrote a prepare script. You may check here.https://github.com/layumi/2016_person_re-ID/blob/master/cuhk03-prepare-eval/prepare_imdb.m#L9

layumi avatar May 31 '18 13:05 layumi

Thanks for your reply! I have performed /cuhk03-prepare-eval/prepare_imdb.m I can not find cuhk03_feature_res_split6.mat rather than cuhk_data_256_split6.mat. cuhk03_feature_res_split6.mat is located in 2016_person_re-ID-master\cuhk03-prepare-eval\eval_cuhk03_10times.m. it is located in first line.I think this file should be stored in the characteristics of resnet-50 extraction

daichenchao avatar May 31 '18 14:05 daichenchao

Sorry to bother you. I have found the cuhk03_feature_res_split6.mat generated file. Very embarrassing, I did not double-check.Thank you again! i have got the rank1, rank5 and map.

daichenchao avatar Jun 01 '18 09:06 daichenchao

sorry to bother you,can you tell me where is the' cuhk03_feature_res_split6.mat',I can not find it.Thank you very much. @daichenchao

rmxhhh avatar Oct 26 '18 12:10 rmxhhh

a function file in document is used to generate cuhk03_feature_res_split6.mat.you can find it carefullly

daichenchao avatar Oct 26 '18 12:10 daichenchao

I found it ,Thank you! @daichenchao

rmxhhh avatar Oct 27 '18 00:10 rmxhhh