How to choose layers to NOT prune?

[aidevmin]

@VainF I want to specify layers that is not pruned. Beside the last layer that related to number of classes, do you have any recommend of not-pruned layers? As my understading, we should keep first layers, because it have large effect on model's performance.

[VainF]

Hi @aidevmin, this is a case-by-case problem. We can only have some empirical designs such as ignoring the first layer during pruning.

[aidevmin]

@VainF Thanks, I think so. We need to keep the last layer related to number of classes and the first layer. I have one more questions. Do you have any recommed for number of epochs when using Group-BN, Group-GReg and your method with sparse training? If this number is larger, it can increase total training time.