DeepSpeedExamples
DeepSpeedExamples copied to clipboard
Add snip_momentum structured pruning example with 80% sparsity ratio
This PR is used to demonstrate the functionality of snip_momentum structured pruning algo implemented in here.
User can reproduce below result by running source ./bash_script/pruning_sparse_snip_momentum.sh with the PR mentioned at above.
| pattern | sparsity ratio | pruning method | epochs | acc & mm-acc |
|---|---|---|---|---|
| 1x1 | 80% | DeepSpeed L1 | 2 | 0.8113/0.822 |
| 1x1 | 80% | Snip_momentum | 2 | 0.8176/0.822 |
| 4x1 | 80% | snip_momentum | 10 | 0.8248/0.8305 |