mmrazor icon indicating copy to clipboard operation
mmrazor copied to clipboard

Published 20 hours ago • verified publisher icon open-mmlab

[Bug] Custom Distillation MMSeg CWD loss nan problem

Open Priyanshu88 opened this issue 10 months ago • 1 comments

Describe the bug

I am training the segnext_l model as a standard teacher on my data and keeping the checkpoint obtained for distillation(mmseg/cwd) from segnext_l--->segnext_tiny. When doing this after starting few iterations i am getting all losses as nan for all upcoming iterations.

I am using all the latest versions .

image

The results of student model also remains 0.

Priyanshu88 avatar Apr 12 '24 07:04 Priyanshu88