anyrl-py icon indicating copy to clipboard operation
anyrl-py copied to clipboard

Zeros in feed_dict cause act and val to be NaN in feed-forward

Open ycps opened this issue 6 years ago • 0 comments

In anyrl/models/feedforward_ac.py, the function step may assign zero-values in feed_dict. This may cause self.session.run((self.actor_out, self.critic_out), feed_dict) to return act and val as arrays containing nan's that may eventually throw an exception.

In order to reproduce the following Traceback, simply modify the examples/cartpole.py, replacing 'CartPole-v0' with 'RoboschoolHumanoid-v1'. (Installing Roboschool and including import roboschool may be necessary)

Traceback (most recent call last):
  File "/home/user/py/run.py", line 72, in <module>
    main()
  File "/home/user/py/cartpole.py", line 24, in main
    run_algorithm(algo)
  File "/home/user/py/cartpole.py", line 49, in run_algorithm
    rollouts = roller.rollouts()
  File "/home/user/py/env/lib64/python3.6/site-packages/anyrl/rollouts/rollers.py", line 48, in rollouts
    model_out = self.model.step([obs], states)
  File "/home/user/py/env/lib64/python3.6/site-packages/anyrl/models/feedforward_ac.py", line 60, in step
    'actions': self.action_dist.sample(act),
  File "/home/user/py/env/lib64/python3.6/site-packages/anyrl/spaces/continuous.py", line 40, in sample
    return np.random.normal(loc=means, scale=stddevs)
  File "mtrand.pyx", line 1656, in mtrand.RandomState.normal
ValueError: scale < 0

ycps avatar Aug 30 '18 02:08 ycps