sparseml
sparseml copied to clipboard
Clean up global RigL sparsities to be exact
Current RigL code uses the param scorer to enforce custom sparsity patterns (Erdos-Renyi, ERK) on the masks. This PR instead passes the ER/ERK sparsity targets directly to the mask creator, to ensure that the mask distribution exactly matches the targets (no rounding errors or other effects).
Tested: did a few runs with these changes and confirmed that the quality matches previous implementation. Updated tests.