finetuning on deepglint, no improvement?

[SueeH]

---

Based on open model(trained on ms1m,test acc are 99.77/98.27/98.28), I finetuning the model on Deepglint, but test acc on lfw/cfp_fp/ageddb are 99.77/96.61/98.15, show above. Besides, I collect several face in real as another testset, this test results also shows the finetuning model has no improvement for face recognition. So anyone did something similar?

-------training logs----------- testing verification.. (12000, 512) infer time 23.246731 [lfw][444000]XNorm: 20.568385 [lfw][444000]Accuracy-Flip: 0.99783+-0.00259 testing verification.. (14000, 512) infer time 26.563688 [cfp_fp][444000]XNorm: 22.903938 [cfp_fp][444000]Accuracy-Flip: 0.96614+-0.01038 testing verification.. (12000, 512) infer time 22.508262 [agedb_30][444000]XNorm: 23.183821 [agedb_30][444000]Accuracy-Flip: 0.98150+-0.00736 saving 111 INFO:root:Saved checkpoint to "../models/model-r100-glint-arc2/model-0111.params" [444000]Accuracy-Highest: 0.98150 INFO:root:Epoch[14] Batch [23820] Speed: 109.18 samples/sec acc=0.981185 INFO:root:Epoch[14] Batch [23880] Speed: 338.00 samples/sec acc=0.980519 INFO:root:Epoch[14] Batch [23940] Speed: 343.30 samples/sec acc=0.980000 INFO:root:Epoch[14] Batch [24000] Speed: 344.53 samples/sec acc=0.982222 INFO:root:Epoch[14] Batch [24060] Speed: 342.03 samples/sec acc=0.982074

---

1.Anyone finetuning the model? Is there any improvement? 2.I checked ms1m has 5.8M img and 85K IDs(65 imgs/id), but deepglint has 6.75M imgs and 181K IDs(37 imgs/id), deepglint has more ids, but the total imgs almost same. So if keep total imgs same(10M), more id(400K,25imgs/id), less id(200K,50imgs/id) which is more useful for face recognition training? 3.Anyone trys to merge ms1m and deepglint and trains model from beginning? any improvement?

Glad to discuss!

[jinwu07]

@SueeH I am thinking about doing the similar thing. How did you exactly fine-tune the model on Deepglint data set?

[SueeH]

@SueeH I am thinking about doing the similar thing. How did you exactly fine-tune the model on Deepglint data set?

I can not get your points exactly, could you show more details？ I just downloaded Deepglint dataset and changed some params, pretrained, lr ...

[jinwu07]

@SueeH I was wondering what command line you use to fine tune. Can you share? Thanks,

[SueeH]

All the params same with authors, I just fine-tune the lr, keep other params the same Since my gpu memory limited, 45 for each is full use of gpu memory.

gpu num: 5 num_layers 100 image_size [112, 112] num_classes 180855 Called with argument: Namespace(batch_size=225, beta=1000.0, beta_freeze=0, beta_min=5.0, bn_mom=0.9, ce_loss=False, ckpt=1, color=0, ctx_num=5, cutoff=0, data_dir='/data/Face/Recg/mxnet/DeepGlint/', easy_margin=0, emb_size=512, end_epoch=500000, fc7_lr_mult=1.0, fc7_no_bias=False, fc7_wd_mult=1.0, gamma=0.12, image_channel=3, image_h=112, image_size='112,112', image_w=112, images_filter=0, loss_type=4, lr=0.1, lr_steps='100000,150000,200000', margin=4, margin_a=1.0, margin_b=0.2, margin_m=0.3, margin_s=64.0, max_steps=0, mom=0.9, network='r100', num_classes=180855, num_layers=100, per_batch_size=45, power=1.0, prefix='../models/model-r100-glint-arc/model', pretrained='../models/model-r100-glint-arc/model,0', rand_mirror=1, rescale_threshold=0, scale=0.9993, target='lfw,cfp_fp,agedb_30', use_deformable=0, verbose=4000, version_act='prelu', version_input=1, version_multiplier=1.0, version_output='E', version_se=0, version_unit=3, wd=0.0005) loading ['../models/model-r100-glint-arc/model', '0'] init resnet 100 0 1 E 3 prelu INFO:root:loading recordio /data/Face/Recg/mxnet/DeepGlint/train.rec... header0 label [6753546. 6934401.] id2range 180855 6753545

[jinwu07]

You were using initial learning rate lr=0.1 with arcface loss, it almost like re-train from scratch. Can you try to set lr = 0.005, and also try triplet loss (use train_triplet.py)?

[jinwu07]

Also, I don't think it is good idea to fine-tune on the full glint data set since there is huge overlap between emore and glint. It is better just fine tune on glintasia dataset.

[SueeH]

You were using initial learning rate lr=0.1 with arcface loss, it almost like re-train from scratch. Can you try to set lr = 0.005, and also try triplet loss (use train_triplet.py)?

1. I have checked when lr=0.1, ran lfw/cfp_fp/agedb evaluation, acc always keeps bigger than 90%, so I think it‘s not re-train. 2.I tried triplet with same params with author, train acc keep 47%, lfw/cfp_fp/agedb about 99.7,93.7,97.7. Low train acc may be caused by softmax layer weight. 3.emore and glint have huge overlap? I have not checked this yet. More infos about these two datasets?

[jinwu07]

After some toying with glint/glint-asia fine tuning, I realized you wouldn't necessarily get better accuracy on validation sets (lfw, cfp_fp, agedb_30) after the fine tune. It should be close to the accuracy before fine tune however. On the other hand, the fine-tuned model has more discriminative power because glint has doubled the identities. In other word, it probably performs better in the real application or other datasets(try megaface or trillion pairs). In your case, your cfg_fp accuracy is a bit off. You were getting very high training accuracy (98%), it sounds to me you were overfitting. Maybe you can play with weight decay (wd) values and see if you can get better results.

[HuanJiML]

@jinwu07 which initial LR have you used? 0.1 or 0.005?

[SueeH]

After some toying with glint/glint-asia fine tuning, I realized you wouldn't necessarily get better accuracy on validation sets (lfw, cfp_fp, agedb_30) after the fine tune. It should be close to the accuracy before fine tune however. On the other hand, the fine-tuned model has more discriminative power because glint has doubled the identities. In other word, it probably performs better in the real application or other datasets(try megaface or trillion pairs). In your case, your cfg_fp accuracy is a bit off. You were getting very high training accuracy (98%), it sounds to me you were overfitting. Maybe you can play with weight decay (wd) values and see if you can getting better results.

Agree with you. Now I finding some other data(include trillion pairs) to evaluate the model.

[jinwu07]

@HuanJiML 0.1, so not exactly "fine-tune".

[eguoguo321]

I've got a better performance on my own dataset,whats U should do is being patient and down your lr to 0.01,0.005 and 0.001

[SueeH]

I've got a better performance on my own dataset,whats U should do is being patient and down your lr to 0.01,0.005 and 0.001

better performance on your own dataset?
I found the acc on lfw, cfp_fp, agedb_30 is high enough(all keep bigger than 98%), so I am not sure these 3 dataset can tell if the new fine-tuned model is better.

[Linzaer]

I use my dataset to train from strach and got acc on lfw, cfp_fp, agedb_30 (99.8 | 99.857 | 96.786 | 98.217) which better than official r100 model.

[XiaYuanxiang]

I use my dataset to train from strach and got acc on lfw, cfp_fp, agedb_30 (99.8 | 99.857 | 96.786 | 98.217) which better than official r100 model.

could you share more details？

[Neltherion]

@Linzaer Could you please elaborate on your dataset? Specifically the scale (how many identities are in your dataset?) and could you please pinpoint the factor that made your model better than the pretrained Resnet50 and Resnet100 models?

Also which architecture did you use for your model? the 50 Layer architecture or the 100 layer?

Thanks

[Linzaer]

@Neltherion About 15w IDs and 1500w imgs.This model (r100) did better on the asian test set .(99.717%>99.667% (official r100) )

[lmolhw5252]

@Linzaer Have you test on a large datasets? What the performance on a large test dataset?

[FengMu1995]

I've got a better performance on my own dataset,whats U should do is being patient and down your lr to 0.01,0.005 and 0.001 问下你用的什么损失函数在预训练的基础上再训练的