Meta-Weight-Net_Code-Optimization icon indicating copy to clipboard operation
Meta-Weight-Net_Code-Optimization copied to clipboard

Published 20 hours ago • verified publisher icon ShiYunyi

How to avoid weight close to zero?

Open shanshuo opened this issue 3 years ago • 0 comments

Thanks for your elegant code. I've got a question about the output weight. Though sigmoid function could output a non-zero vector. https://github.com/ShiYunyi/Meta-Weight-Net_Code-Optimization/blob/1d3cab0f4ad435e2569de97b21694771437321b3/model.py#L108 But the weight vector's elements could be all close to zero, leading to a small loss. https://github.com/ShiYunyi/Meta-Weight-Net_Code-Optimization/blob/1d3cab0f4ad435e2569de97b21694771437321b3/main.py#L129 How could we avoid this?

shanshuo avatar Feb 20 '22 16:02 shanshuo