GoT-GTRL

:page_with_curl: Goal-guided Transformer-enabled Reinforcement Learning for Efficient Autonomous Navigation

[Published Paper] | [arXiv] | [BiliBili] | [Youtube]

:dizzy: A goal-driven mapless end-to-end autonomous navigation of unmanned grounded vehicle (UGV) realized through Transformer-enabled deep reinforcement learning (DRL) algorithm.

:blue_car: A car-like mobile robot learns to autonomously navigate to a random goal position only through raw RGB images from one Fisheye camera and goal information in polar coordination system.

:wrench: Realized in ROS Gazebo simulator with Ubuntu 20.04, ROS noetic, and Pytorch.

Citation

If you find this repository useful for your research, please consider starring :star: our repo and citing our paper.

@ARTICLE{huang2023goal,
  author={Huang, Wenhui and Zhou, Yanxin and He, Xiangkun and Lv, Chen},
  journal={IEEE Transactions on Intelligent Transportation Systems}, 
  title={Goal-Guided Transformer-Enabled Reinforcement Learning for Efficient Autonomous Navigation}, 
  year={2023},
  volume={},
  number={},
  pages={1-14},
  doi={10.1109/TITS.2023.3312453}}

Preview Simulation

Click the gif to zoom in :mag_right:

Video: Sim-to-Real Experiment :arrow_lower_left:

:point_right: GTRL Sim-to-Real Navigation Experiment Video :point_left:

Basic Dependency Installation

:one: ROS Noetic

:two: Gazebo

:three: Pytorch

User Guidance

Create a new Virtual environment (conda is suggested).

Specify your own name for the virtual environment, e.g., gtrl:

conda create -n gtrl python=3.7

Activate virtual environment.

conda activate gtrl

Install Dependencies.

pip install numpy tqdm natsort cpprb matplotlib einops squaternion opencv-python rospkg rosnumpy yaml
sudo apt install python3-catkin-tools python3-osrf-pycommon
sudo apt-get install ros-noetic-cv-bridge

Optional step for visualizing real-time plotting (reward curve) with Spyder.

conda install spyder==5.2.2

Clone the repository.

cd to your workspace and clone the repo.

git clone https://github.com/OscarHuangWind/DRL-Transformer-SimtoReal-Navigation.git

Compile the workspace.

cd ~/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws

catkin_make -DPYTHON_EXECUTABLE=/usr/bin/python3

Set up the environment variables.

export GAZEBO_RESOURCE_PATH=~/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/gtrl/launch
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/ros/noetic/lib

Alternatively, you can select to write these variables to the ~/.bashrc file so that it can be automatically set when opening terminal.

Source the workspace.

source devel/setup.bash

Important!

Copy all the files under models folder to your default gazebo models folder.

cp -a ~/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/gtrl/models/. ~/.gazebo/models

Revise your system path in main.py and env_lab.py (gtrl/scripts/Environments/env_lab.py) file.

main.py

import sys
sys.path.append('/home/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/gtrl/scripts')

env_lab.py (line 129)

fullpath = os.path.join('/home/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/drl_navigation/launch', launchfile)

Time to train and get your GTRL model!!!

cd ~/$your workspace/DRL-Transformer-SimtoReal-Navigation/catkin_ws/src/gtrl/scripts/SAC

Run it in the terminal:

python main.py

(Optional) Alternatively, if you have already installed spyder, just click the run file button in spyder.

To kill the program, it is suggested to use following commands.

killall -9 rosout roslaunch rosmaster gzserver nodelet robot_state_publisher gzclient python python3

Alternatively, you can add alias of these commands to the ~/.bashrc file:

alias k9='killall -9 rosout roslaunch rosmaster gzserver nodelet robot_state_publisher gzclient python python3'

And type the alias in the terminal to kill all the process:

k9

Framework

Goal-guided Transformer (GoT)

Noise-augmented RGB images from fisheye camera

AGV and lab environment model in simulation and real world.

Sim-to-Real navigaiton experiment in office environment.