Tommy Tao

icon indicating an email [email protected]

icon company Hospital Authority icon location Hong Kong icon location AI Developer with full-stack development abilities, MLOps management experience and research mindset on NLP, ASR and medical computer vision

Results 13 comments of Tommy Tao

First of all, thank you for sharing. You say, "This inversion only calculates derivatives, so it is not considered backpropagation. Hinton points out that you don't need to know the specifics to derive. Isn't this a step against the point in the paper? Thank you.

Other estimation/optimization methods that do not require gradients

If there is a black box then how to update the parameters

I am confused about it too. Maybe, there is an analog hardware component which could maximize its magnitude of output vector under a certain input through updating its internal states...

Why use Softplus function in loss?

I observed followings: 1. Both serve the same purpose, maximize g_pos and minimize g_neg. 2. Original FF gain/loss range is (0, 1) with near-zero gradients around both 0 and 1...