Limited prunable parameters (like shortcut, ..)

[kiddj]

Uploaded codes in repo does not seemed to support some parameters such as shortcut, batch-norm and bias terms.

Does LAP only work on weights except above terms?

[jaeho-lee]

Hi-

thank you for your interest in our work!

Our current repo only supports pruning weights (which dominate the number of parameters), but our framework can be smoothly extended to pruning shortcuts and batch-norms. The framework neglects bias terms.

We plan to make some updates -- after NeurIPS deadline -- to our codes to reflect the updates in the PyTorch 1.5.

Notes about pruning bias: There are some reasons to believe that either removing them all at once, or keeping them altogether won't be too detrimental to the performance. I like the discussion here: https://www.reddit.com/r/MachineLearning/comments/eymex9/d_why_isnt_the_bias_terms_in_the_weights_also/

We'll keep this issue opened until we update our package.