Freezing Pruned weights smaller than 1e-6

[yuezhixiong]

In main.py #Freezing Pruned weights by making their gradients Zero grad_tensor = np.where(tensor < EPS, 0, grad_tensor)

Does this also freeze the weights that have negative values? More than just weights with 0 values?

[xiaosaisworking]

I think it's really a bug. Once solving this problem, my experiment results can reach the original performance in the first iteration(pruning has not started). Hope to help you!

[JaminLi]

I think it's really a bug. Once solving this problem, my experiment results can reach the original performance in the first iteration(pruning has not started). Hope to help you!

I also meet this problem, my experiment results can't reach the original performance. How can I solve this problem with the "grad_tensor = np.where(tensor < EPS, 0, grad_tensor)" , please?

[zdhNarsil]

@JaminLi I think it should be grad_tensor = np.where(tensor.abs() < EPS, 0, grad_tensor).