Reinforce Policy Update wrong

Open karanchahal opened this issue 6 years ago • 1 comments

The update step for REINFORCE appears to be wrong. Instead of multiplying log probs and rewards at each step, we should sum up all the rewards to go, and the log probs across the entire episode. After getting these sums, we should multiply these 2 values together. This was mentioned in Sergey Levine's lecture on Policy gradients where he derived the entire policy gradient algorithm.

Source: https://www.youtube.com/watch?v=Ds1trXd6pos&list=PLkFD6_40KJIwhWJpGazJ9VSj9CFMkb79A&index=5

Feb 28 '20 03:02 karanchahal