Gazebo environment integration

[Privilger]

• [ ] I have marked all applicable categories:
  • [ ] exception-raising bug
  • [ ] RL algorithm bug
  • [x] documentation request (i.e. "X is missing from the documentation.")
  • [x] new feature request
• [x] I have visited the source website, and in particular read the known issues
• [x] I have searched through the issue tracker for duplicates
• [x] I have mentioned version numbers, operating system and environment: 0.2.1 3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54) [GCC 7.3.0] linux

Hello there, I'm using the PPO algorithm in this library.

I'm reading the examples/pong_ppo.py. In line 52, it is using the SubprocVectorEnv to create a lot of parallel environment.

Gazebo is a popular simulator in robotics, which supports ROS. I'm using the Gazebo as my environment and run robot simulation. What should I do to make my environment support this SubprocVectorEnv function. I already make the Gazebo works on one environment instance.

[Trinkle23897]

You can use VectorEnv first.

BTW, I'm not sure whether pong_ppo.py can work well (typically for hyper-parameter tuning). @Mehooz has other deadlines (2 NeurIPS papers), he will resolve this issue once he finishes his deadlines.

[Trinkle23897]

Another solution is to create a small number of env (e.g. num = 4), for test:

envs = SubprocVectorEnv([env_fns for i in range(num)])
test_collector = Collector(policy, envs)

and pass episode_per_test=100 into the trainer, you will see each environment run 25 episodes.

[Privilger]

OK, thanks a lot!

[duburcqa]

@Privilger I just wanted to let you know I developped a robotic simulator called Jiminy that is much faster, light weight, and easy to use than Gazebo. It is already integrated with OpenAI Gym and I provide an exemple for Cartpole policy learning with PPO algorithm using several frameworks, including Tianshou.

[Trinkle23897]

@duburcqa That's great!

[duburcqa]

@Trinkle23897 By the way, I would like to thank you for the amazing work that you did for Tianshou. In my opinion, it is by far the best framework for quick prototyping (I'm not saying it is the case for production, since ray[rllib] is very impressive on this point, but it is an issue for only few people).

[Tortes]

Here is my Gazebo environment for PPO. I hope it will help u.

[sumingzs11]

However, the gazebo simulator cannot be opened in both directions at the same time. I want to know how you apply Tianshou to gazebo Is the test part removed by modifying the training strategy

[MischaPanch]

Closing due to inactivity