Conscious-Planning
Conscious-Planning copied to clipboard
Implementation for paper "A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning".
A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning
By Mingde "Harry" Zhao, Zhen Liu, Sitao Luan, Shuyuan Zhang, Doina Precup and Yoshua Bengio
(BLOGPOST)
Install Dependencies
pip install -r requirements.txt
Reproducing Results
CP
python run_distshift_randomized_mp.py --method DQN_CP --num_explorers 8 --ignore_model 0 --disable_bottleneck 0 --size_bottleneck 8
UP
python run_distshift_randomized_mp.py --method DQN_CP --num_explorers 8 --ignore_model 0 --disable_bottleneck 1
WM
python run_distshift_randomized_mp.py --method DQN_WM --num_explorers 8 --ignore_model 0 --disable_bottleneck 0 --size_bottleneck 8 --period_warmup 1000000
Dyna
python run_distshift_randomized_mp.py --prioritized_replay 0 --method DQN_Dyna --num_explorers 8 --ignore_model 0 --disable_bottleneck 0 --size_bottleneck 8 --learn_dyna_model 1
Special thanks to my colleague and friend Safa Alver @alversafa for pointing out that Dyna should not use prioritized buffer as it shouldn't prioritize on the errors generated by potentially inaccurate imagined transitions, as well as the runtime bugs surrounding this matter!
Dyna*
python run_distshift_randomized_mp.py --method DQN_Dyna --num_explorers 8 --ignore_model 0 --disable_bottleneck 0 --size_bottleneck 8 --learn_dyna_model 0
NOSET
python run_distshift_randomized_mp.py --method DQN_NOSET --num_explorers 8 --ignore_model 0 --layers_model 2 --len_hidden 256
Changing Settings
Read run_distshift_randomized_mp.py!