Deep-reinforcement-learning-with-pytorch icon indicating copy to clipboard operation
Deep-reinforcement-learning-with-pytorch copied to clipboard

Published 20 hours ago • verified publisher icon sweetice

Bugs in PPO

Open moonblue333 opened this issue 5 years ago • 7 comments

  1. counter

  2. for index in BatchSampler(SubsetRandomSampler(range(self.buffer_capacity), self.batch_size, True)):

moonblue333 avatar May 05 '19 15:05 moonblue333

  1. How to solve bug 2?
  2. It seems that "done" from the env hasn't been dealt with. Why?

Thanks.

yuntao-ma avatar Nov 25 '19 11:11 yuntao-ma

@yuntao-ma for index in BatchSampler(SubsetRandomSampler(range(self.buffer_capacity)), self.batch_size, True):

HuangHaoyu1997 avatar Mar 08 '20 17:03 HuangHaoyu1997

Why can I train with this code to only get nan actions

brezezee avatar Jul 27 '20 13:07 brezezee

I get nan actions too

xxx-007 avatar Nov 04 '20 01:11 xxx-007

I change the code to : for index in BatchSampler(SubsetRandomSampler(range(self.buffer_capacity)), self.batch_size, True): but there still exists a bug: Traceback (most recent call last): File "E:/AAAFor_PHD/UUV_SCI_Modif/UUV_obs_env/PPO2/Demo/PPO_demo.py", line 195, in <module> main() File "E:/AAAFor_PHD/UUV_SCI_Modif/UUV_obs_env/PPO2/Demo/PPO_demo.py", line 175, in main next_state, reward, done, info = env.step(action) File "F:\Anaconda\envs\Obstacle_Avoid\lib\site-packages\gym\envs\classic_control\pendulum.py", line 49, in step u = np.clip(u, -self.max_torque, self.max_torque)[0] IndexError: invalid index to scalar variable.

HzcIrving avatar Nov 28 '20 09:11 HzcIrving

Transition = namedtuple('Transition',['state', 'aciton', 'reward', 'a_log_prob', 'next_state']) 'aciton' should be 'action'

haohaoqian avatar Dec 13 '23 04:12 haohaoqian