Receptive-Field-in-Pytorch icon indicating copy to clipboard operation
Receptive-Field-in-Pytorch copied to clipboard

Published 20 hours ago • verified publisher icon rogertrullo

Sigmoid Activation

Open filipafcastro opened this issue 5 years ago • 0 comments

Hi @rogertrullo,

Thanks for sharing this. While computing the receptive field of a network with a sigmoid in the forward pass, the gradient does not flow as expected, probably because the gradient vanishes. I think it may then be useful to add something like "Turn off / Remove sigmoid activations" as a requirement. Or did I interpret it wrongly? What do you think?

filipafcastro avatar Dec 27 '19 16:12 filipafcastro