examples icon indicating copy to clipboard operation
examples copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

Reinforce Policy Update wrong

Open karanchahal opened this issue 5 years ago • 1 comments

The update step for REINFORCE appears to be wrong. Instead of multiplying log probs and rewards at each step, we should sum up all the rewards to go, and the log probs across the entire episode. After getting these sums, we should multiply these 2 values together. This was mentioned in Sergey Levine's lecture on Policy gradients where he derived the entire policy gradient algorithm.

Source: https://www.youtube.com/watch?v=Ds1trXd6pos&list=PLkFD6_40KJIwhWJpGazJ9VSj9CFMkb79A&index=5

karanchahal avatar Feb 28 '20 03:02 karanchahal