d2l-en
d2l-en copied to clipboard
Analogy that explains intuition of gradient descent
I think this book should also contain analogy where we need to descend from the top of the hill down to the valley. The only difference between what I usually encountered over the internet and this one is the addition of the fog. For me it makes a huge difference and helps to understand the intuition behind gradient descent. If we cannot see what's around us, only what is underneath, that explains why:
- why we need to take small steps
- why we can stack at local minima (we stop when the slop is equal to zero but that doesn't mean that this is the lowest point)
- why we might start moving in a wrong direction because we don't know if after ascend there will be descent which will lead us to the lowest point
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Job d2l-en/PR-2403/1 is complete. Check the results at http://preview.d2l.ai/d2l-en/PR-2403/