ReviewKD
ReviewKD copied to clipboard
shapes and out_shapes values in ReviewKD
To me, it's confusing, how to set the "shapes" and "out_shapes" when the student is ResNet18 and the teacher is ResNet34 on CIFAR-100.
Is it shapes = out_shapes = [1, 8, 16, 32, 32]
or, shapes = out_shapes = [1, 4, 8, 16, 32]
It depends on the shape of features you used for distillation.