mmfewshot icon indicating copy to clipboard operation
mmfewshot copied to clipboard

test query set would have some samples from test support set

Open benjohn18 opened this issue 2 years ago • 1 comments

I reload your model from metafile.yml and want to test the performance of MMfewshot in my dataset. I found that when I input dataset with different numbers of images like 50, 100 ,200 ,400 , the more images I input , the less acc I got. And I found that datawraper would load same images into support set and query set . The less images I input ,the higher acc is. There is my result.

model=meta-baseline                
  800 eposide              
  num querys 15              
  5 way 5shot   5 way 10shot   5 way 15shot   5 way 20shot  
input_pics acc acc std acc acc std acc acc std acc acc std
50 69.7 0.46 75.91 0.43 78.05 0.4 79.6 0.36
100 63.77 0.47 69.33 0.41 71.22 0.38 72.4 0.36
200 60.63 0.43 66.48 0.41 68.59 0.4 70.24 0.4
400 59.76 0.43 65.44 0.4 67.5 0.38 68.93 0.38

benjohn18 avatar Apr 19 '22 03:04 benjohn18

The model will replace the classifier head and finetune it with N-way K-shot setting while testing. 1 eposide = 5way*(5support+15querys)=100pictures for your setting 5shot/10shot/15shot/20shot, the number of picture is not enough

HengYuD avatar Jun 21 '22 03:06 HengYuD