Prioritized Human-In-the-Loop (PHIL) Reinforcement Learning for Behavior Planning of Autonomous Vehicles in CARLA

This repo is the implementation of the following paper:

Prioritized Experience-Based Reinforcement Learning With Human Guidance for Autonomous Driving
Jingda Wu, Zhiyu Huang, Wenhui Huang, Chen Lv

AutoMan Research Lab, Nanyang Technological University

[arXiv] [Project Website]

Getting started

1. Install the CARLA simulator, with referring to https://carla.readthedocs.io/en/latest/start_quickstart/#a-debian-carla-installation

2. Install the dependent package

pip install -r requirements.txt

3. Training the RL agent in the left-turn scenario

python train_leftturn.py
--algorithm 0 
--human_model 
--reward_shaping 0 
--seed 123 
--maximum_episode 400 
--initial_exploration_rate 0.5 
--cutoff_exploration_rate 0.05 
--exploration_decay_rate 0.99988 
--warmup 
--warmup_threshold 1e4 
--device cuda 
--simulator_port 2000 
--simulator_render_frequency 12 
--joystick_enabled

4. Training the RL agent in the congestion scenario

python train_congestion.py
--algorithm 0 
--human_model 
--human_model_type PI 
--reward_shaping 0 
--seed 123 
--maximum_episode 400 
--initial_exploration_rate 0.5 
--cutoff_exploration_rate 0.05 
--exploration_decay_rate 0.99988 
--warmup 
--warmup_threshold 1e4 
--device cuda 
--simulator_port 2000 
--simulator_render_frequency 12 
--joystick_enabled

Using human guidance to improve RL performance (by joystick or keyboard)

The algorithms in this repo allow human subjects to provide guidance for improving RL training performance in the real-time (by joystick or keyboard)

Training performance

(a-b) Results in the left-turn scenario; (c-d) Results in the congestion scenario.

Reference

If you find this repo to be useful in your research, please consider citing our work

@ARTICLE{9793564,
  author={Wu, Jingda and Huang, Zhiyu and Huang, Wenhui and Lv, Chen},
  journal={IEEE Transactions on Neural Networks and Learning Systems}, 
  title={Prioritized Experience-Based Reinforcement Learning With Human Guidance for Autonomous Driving}, 
  year={2022},
  doi={10.1109/TNNLS.2022.3177685}}

License

This repo is released under GNU GPLv3 License.