Allow to setup compression modules for the training-only modules

Open ljaljushkin opened this issue 4 years ago • 1 comments

It's needed for elastic width in case of Aux branches, e.g. in InceptionV3. We still don't need to compress these branches, but we need Mixin modules with pre_ops to propagate changes in the width.

Suggested approach - simple replace eval graph with train one and mark all training-only nodes to filter them in all previous algorithms.

Why is it draft? There're some unresolved issues with statistics and setting compression. Looks like some compression is set for training only layers. Also wrapping training-only layers leads to some issues with LSTM.

Jul 22 '21 14:07 ljaljushkin

Jenkins please stop all builds

Jul 27 '21 07:07 ljaljushkin