PaddleFL icon indicating copy to clipboard operation
PaddleFL copied to clipboard

using loss scaling to improve model accuracy

Open honshj opened this issue 4 years ago • 0 comments

Due the 64bit fixed-point type used in MPC, the smallest number is 2^-16. As a result, gradients less than 2^-16 during backward pass will become zeros, which degrades the accuracy of the result model. A technique like loss scaling (https://arxiv.org/abs/1710.03740) can be used to alleviate this issue.

honshj avatar Jul 30 '20 03:07 honshj