knowledge-distillation-pytorch
knowledge-distillation-pytorch copied to clipboard
haitongli
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
I'm unable to run train.py on python 3.9. The versions stated in requirements are wrong, and after installing the newest libraries there's a bunch of syntax errors in the program....
Bumps [numpy](https://github.com/numpy/numpy) from 1.14.0 to 1.22.0. Release notes Sourced from numpy's releases. v1.22.0 NumPy 1.22.0 Release Notes NumPy 1.22.0 is a big release featuring the work of 153 contributors spread...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 1.7.0rc0 to 2.7.2. Release notes Sourced from tensorflow's releases. TensorFlow 2.7.2 Release 2.7.2 This releases introduces several vulnerability fixes: Fixes a code injection in saved_model_cli (CVE-2022-29216) Fixes...
Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.1.1 to 9.0.1. Release notes Sourced from pillow's releases. 9.0.1 https://pillow.readthedocs.io/en/stable/releasenotes/9.0.1.html Changes In show_file, use os.remove to remove temporary images. CVE-2022-24303 #6010 [@radarhere, @hugovk] Restrict builtins within...
I suggest both training loss function without KD and with KD should add a softmax function, because the outputs of models are without softmax. Just like this. https://github.com/peterliht/knowledge-distillation-pytorch/blob/e4c40132fed5a45e39a6ef7a77b15e5d389186f8/model/net.py#L100-L114 ==> `KD_loss...
Hi @peterliht , Thanks for you great job! I am trying to train my own dataset , however I got RuntimeError: size mismatch, m1: [2 x 2048], m2: [512 x...
Hello peterliht， I ran through your code according to the instructions, did not modify any parameters, but found that the results vary greatly. What parameters did you modify before releasing...
Teacher model's outputs are only computed before training epoch. https://github.com/peterliht/knowledge-distillation-pytorch/blob/master/train.py#L277 It assumes that inputs are fixed in each epoch. But the inputs are different in each epoch due to the...
I can't download the box folder.Could someone send these files to my mailbox?Thank you so much!
