DeepSpeedExamples
DeepSpeedExamples copied to clipboard
Add snip_momentum structured pruning example with 80% sparsity ratio
This PR is used to demonstrate the functionality of snip_momentum structured pruning algo implemented in here.
User can reproduce below result by running source ./bash_script/pruning_sparse_snip_momentum.sh
with the PR mentioned at above.
pattern | sparsity ratio | pruning method | epochs | acc & mm-acc |
---|---|---|---|---|
1x1 | 80% | DeepSpeed L1 | 2 | 0.8113/0.822 |
1x1 | 80% | Snip_momentum | 2 | 0.8176/0.822 |
4x1 | 80% | snip_momentum | 10 | 0.8248/0.8305 |