sparseml icon indicating copy to clipboard operation
sparseml copied to clipboard

Published 20 hours ago • verified publisher icon neuralmagic

Preserve sparsity GPTQ

Open rahul-tuli opened this issue 1 year ago • 0 comments

Recently a bug was revealed, where if GPTQ modifier was applied consecutively after SparseGPT, the weight sparsity mask was not being respected, this PR fixes that by preserving the mask, we do this automatically if the weight sparsity is greater than SPARSITY_THRESHOLD which has been set to 5% for now.

Credits to @Satrat and @abhinavnmagic for proposing the fix

The unit test for consecutive application now runs w/o having to increase the relative tolerance which was done as a part of #2272

rahul-tuli avatar May 13 '24 13:05 rahul-tuli