VirtualtoReal-RL
VirtualtoReal-RL copied to clipboard
xinleipan
How to use this TORCS in reinforcement learning ?
I notice that image is available through shared memory with C/C++. But for reinforcement learning, I also need to send control instruction to it. What's more, it's better to get image via python, since python is more popular in RL.
Regular version of gym-TORCS only supports 64 * 64 image, which size is too small .
hi @18605973470 , sorry for my late reply. Please refer to https://github.com/xinleipan/py_TORCS for a version of driving environment compatible with python, it's more flexible than this one.
hi @xinleipan
I can run py_TORCS without crash only under a few tracks (eg. michigan). Many tracks may cause a segmentation fault. I don't konw why.
/usr/local/bin/torcs: line 53: 13327 Segmentation fault (core dumped) $LIBDIR/torcs-bin -l $LOCAL_CONF -L $LIBDIR -D $DATADIR $*
@18605973470 that is because we want to be able to provide the segmentation for that environment. We didn't do this for other environments, only the michigan way environment now. If you want to provide the segmentation for other environments, you will need to modify the code accordingly, I'll try to provide the instruction on how to modify the code
@xinleipan
I am looking forward to the guidance to add more customized tracks 👍 I really need it ! :)
@xinleipan
In addition, I also find that it's hard to train with continuous action using DDPG or A3C for image as only input. Have you ever try with continuous action ?
By the way, I found that the car runs very fast when I set throttle only as 0.05, while it's speed is reasonable in gym_torcs.
@xinleipan @XiaoZzai I want to implement the experiment "virturl to real-RL".I found the above version link is invalid.Is there any open source code for reinforcement learning of image input? Thanks.
@lingyunli1994 sorry about that, the environment was (not sure why) set to private. Now it is public!
@xinleipan Thank you very much for your answer. I notice that you have selected 1762 images in the 45k real images, how to selected? Is it randomly selected? Is the selected image associated?
@lingyunli1994 since segnets was originally trained on another driving data, their performance on the driving data I got was not very satisfying, so we picked up the best data with high quality segmentations. Ideally you should have humans label the data to make sure the training data you get is clear enough.
@XiaoZzai we didn't try with DDPG with continuous control and you are more than welcome to give a try! If you are also interested in model-based continuous control, our new paper open sourced here provide the code to train a continuous control policy: https://github.com/ucbdrive/spc