d2l-en icon indicating copy to clipboard operation
d2l-en copied to clipboard

Analogy that explains intuition of gradient descent

Open Andrei-Aksionov opened this issue 2 years ago • 1 comments

I think this book should also contain analogy where we need to descend from the top of the hill down to the valley. The only difference between what I usually encountered over the internet and this one is the addition of the fog. For me it makes a huge difference and helps to understand the intuition behind gradient descent. If we cannot see what's around us, only what is underneath, that explains why:

  1. why we need to take small steps
  2. why we can stack at local minima (we stop when the slop is equal to zero but that doesn't mean that this is the lowest point)
  3. why we might start moving in a wrong direction because we don't know if after ascend there will be descent which will lead us to the lowest point

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Andrei-Aksionov avatar Dec 12 '22 16:12 Andrei-Aksionov

Job d2l-en/PR-2403/1 is complete. Check the results at http://preview.d2l.ai/d2l-en/PR-2403/

d2l-bot avatar Dec 12 '22 16:12 d2l-bot