sparseml
sparseml copied to clipboard
Preserve sparsity SPARSEGPT
This PR incorporates changes from @abhinavnmagic's PR https://github.com/neuralmagic/sparseml/pull/2222 into new modifier UX
We introduce a new argument preserve_sparsity_mask in SparseGPTModifier which can be used to extend or ignore the base weight mask during SparseGPT application
Test Recipe:
test_stage:
obcq_modifiers:
SparseGPTModifier:
sparsity: 0.5
block_size: 128
percdamp: 0.01
mask_structure: "2:4"
targets: ["re:model.layers.\\d*$"]
preserve_sparsity_mask: true