skgyu
runtime too long
Hi there,
thank you for your nice work and code.
I am trying to run the train_cvpr2023.py --config scripts/DFKD_Cifar10_ResNet34_ResNet18_V37_conv012.yaml example on an A100 GPU. With tqdm, the estimated time is about 62 hrs. I feel there maybe something wrong in my environment. May I know how many hours usually do you need to finish this example?
BR,
I checked the last running record of the release version code, and based on the file generation time, the CIFAR100 experiment took approximately 88 hours on a 3090 GPU. The GPU time required for the version submitted for the paper is quite similar, so your observation is correct. This code requires more runtime than some previous DFKD methods . You could also try to identify where optimization might be possible (I’ve been quite busy recently, so I haven’t had much time to look into this.). I suspect the issue may be related to the lmdb python library or its Python bindings.
If you are racing against a paper deadline and need to report results of our method, in addition to referencing the results we reported in our paper [2] or the results you obtained from running our code, you can also refer to [1], which was published in Scientific Reports (a journal published by Nature Publishing Group). The authors of that paper have also ran the code themselves and reported their test accuracies on some of the same datasets we used. Their actual results from running our code are quite similar to what we reported in our paper [2], with some experiments showing slightly higher results, and others slightly lower.
I would like to take this opportunity to thank the researchers who have shown interest in our work.
[1] Pengchen Liang, Jianguo Chen, Yan Wu, Bin Pu, Haishan Huang, Qing Chang, and Guo Ran. Data free knowledge distillation with feature synthesis and spatial consistency for image analysis[J]. Scientific Reports, 2024, 14(1): 27557. [2] Shikang Yu, Jiachen Chen, Hu Han, and Shuqiang Jiang. Data-free knowledge distillation via feature exchange and activation region constraint[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 24266-24275.
I checked the last running record of the release version code, and based on the file generation time, the CIFAR100 experiment took approximately 88 hours on a 3090 GPU. The GPU time required for the version submitted for the paper is quite similar, so your observation is correct. This code requires more runtime than some previous DFKD methods . You could also try to identify where optimization might be possible (I’ve been quite busy recently, so I haven’t had much time to look into this.). I suspect the issue may be related to the lmdb python library or its Python bindings.
If you are racing against a paper deadline and need to report results of our method, in addition to referencing the results we reported in our paper [2] or the results you obtained from running our code, you can also refer to [1], which was published in Scientific Reports (a journal published by Nature Publishing Group). The authors of that paper have also ran the code themselves and reported their test accuracies on some of the same datasets we used. Their actual results from running our code are quite similar to what we reported in our paper [2], with some experiments showing slightly higher results, and others slightly lower.
I would like to take this opportunity to thank the researchers who have shown interest in our work.
[1] Pengchen Liang, Jianguo Chen, Yan Wu, Bin Pu, Haishan Huang, Qing Chang, and Guo Ran. Data free knowledge distillation with feature synthesis and spatial consistency for image analysis[J]. Scientific Reports, 2024, 14(1): 27557. [2] Shikang Yu, Jiachen Chen, Hu Han, and Shuqiang Jiang. Data-free knowledge distillation via feature exchange and activation region constraint[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 24266-24275.
Thank you a lot for your detailed and timely response.
BR,