mmrazor
mmrazor copied to clipboard
detach output from teacher
When using Self Distiller with Channel Wise Distill, there would be a backward exception due to the tensor from teacher is not detached.
The PR fixed this bug.
Or we can detach the tensor in cwd loss (https://github.com/open-mmlab/mmrazor/blob/master/mmrazor/models/losses/cwd.py) , what do you think?
I'm so sorry, I haven't found this PR due to my negligence.
There are some users who will train both teacher and student at the same time.
Maybe adding a flag in SelfDistiller is better.
In SingleTeacherDistiller
, use teacher_trainable
to control.
https://github.com/open-mmlab/mmrazor/blob/e4e95131a2451f748923fe702c269d02c9a753f6/mmrazor/models/distillers/single_teacher.py#L172-L176
Would you like to reopen this pr and complete this feature together?
yup it would be better.
Codecov Report
Merging #100 (c19a8ec) into master (0dd407a) will increase coverage by
9.72%
. The diff coverage is100.00%
.
@@ Coverage Diff @@
## master #100 +/- ##
==========================================
+ Coverage 56.58% 66.31% +9.72%
==========================================
Files 83 92 +9
Lines 2932 3369 +437
Branches 540 613 +73
==========================================
+ Hits 1659 2234 +575
+ Misses 1197 1033 -164
- Partials 76 102 +26
Flag | Coverage Δ | |
---|---|---|
unittests | 66.28% <100.00%> (+9.69%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
Impacted Files | Coverage Δ | |
---|---|---|
mmrazor/models/distillers/self_distiller.py | 100.00% <100.00%> (ø) |
|
mmrazor/utils/__init__.py | 100.00% <0.00%> (ø) |
|
mmrazor/models/ops/__init__.py | 100.00% <0.00%> (ø) |
|
mmrazor/models/losses/__init__.py | 100.00% <0.00%> (ø) |
|
mmrazor/models/ops/darts_series.py | 81.72% <0.00%> (ø) |
|
...els/architectures/components/backbones/__init__.py | 100.00% <0.00%> (ø) |
|
.../architectures/components/heads/no_bias_fc_head.py | 0.00% <0.00%> (ø) |
|
...chitectures/components/backbones/darts_backbone.py | 88.63% <0.00%> (ø) |
|
mmrazor/models/ops/mobilenet_series.py | 95.65% <0.00%> (ø) |
|
mmrazor/apis/mmdet/inference.py | 51.51% <0.00%> (ø) |
|
... and 47 more |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 0dd407a...c19a8ec. Read the comment docs.