sparseml icon indicating copy to clipboard operation
sparseml copied to clipboard

Preserve sparsity SPARSEGPT

Open rahul-tuli opened this issue 1 year ago • 0 comments

This PR incorporates changes from @abhinavnmagic's PR https://github.com/neuralmagic/sparseml/pull/2222 into new modifier UX

We introduce a new argument preserve_sparsity_mask in SparseGPTModifier which can be used to extend or ignore the base weight mask during SparseGPT application

Test Recipe:

test_stage:
  obcq_modifiers:
    SparseGPTModifier:
      sparsity: 0.5
      block_size: 128
      percdamp: 0.01
      mask_structure: "2:4"
      targets: ["re:model.layers.\\d*$"]
      preserve_sparsity_mask: true

rahul-tuli avatar May 13 '24 14:05 rahul-tuli