sparseml icon indicating copy to clipboard operation
sparseml copied to clipboard

Published 20 hours ago • verified publisher icon neuralmagic

TopKast additional tests + bugfix

Open ohaijen opened this issue 1 year ago • 0 comments

Additional tests to ensure Top-KAST is working as intended.

Bugfix: when computing weight decay for the backwards-only weights (set B in the paper), the multiplier should be proportional to 1/(the number of dense weights), not (1/the sparsity).

ohaijen avatar Jan 04 '24 10:01 ohaijen