Awesome In-Context Reinforcement Learning

This is a collection of research papers for In-Context Reinforcement Learning (ICRL). The repository shall be regularly updated to track the frontiers.

Curated by dunnolab.

Please, feel free to PR new papers and resources you believe are relevant and awesome.

format:
- [title](paper link)
  - author1, author2, and author3...

Papers

• In-context Reinforcement Learning with Algorithm Distillation
  • Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, Volodymyr Mnih
• Supervised Pretraining Can Learn In-Context Reinforcement Learning
  • Jonathan N. Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, Emma Brunskill
• Emergence of In-Context Reinforcement Learning from Noise Distillation
  • Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin, Viacheslav Sinii, Sergey Kolesnikov
• In-Context Reinforcement Learning for Variable Action Spaces
  • Viacheslav Sinii, Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman, Sergey Kolesnikov
• In-Context Reinforcement Learning Without Optimal Action Labels
  • Juncheng Dong, Moyang Guo, Ethan X Fang, Zhuoran Yang, Vahid Tarokh
• XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning
  • Alexander Nikulin, Ilya Zisman, Alexey Zemtsov, Viacheslav Sinii, Vladislav Kurenkov, Sergey Kolesnikov
• ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI
  • Ahmad Elawady, Gunjan Chhablani, Ram Ramrakhya, Karmesh Yadav, Dhruv Batra, Zsolt Kira, Andrew Szot
• Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models
  • Can Demircan, Tankred Saanum, Akshay K. Jagadish, Marcel Binz, Eric Schulz
• AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents
  • Jake Grigsby, Linxi Fan, Yuke Zhu
• AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers
  • Jake Grigsby, Justin Sasek, Samyak Parajuli, Daniel Adebi, Amy Zhang, Yuke Zhu
• Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining
  • Licong Lin, Yu Bai, Song Mei
• Structured State Space Models for In-Context Reinforcement Learning
  • Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob Foerster, Satinder Singh, Feryal Behbahani
• LLMs Are In-Context Reinforcement Learners
  • Giovanni Monea, Antoine Bosselut, Kianté Brantley, Yoav Artzi
• EVOLvE: Evaluating and Optimizing LLMs For Exploration
  • Allen Nie, Yi Su, Bo Chang, Jonathan N. Lee, Ed H. Chi, Quoc V. Le, Minmin Chen
• In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
  • Sili Huang, Jifeng Hu, Hechang Chen, Lichao Sun, Bo Yang
• Retrieval-Augmented Decision Transformer: External Memory for In-context RL
  • Thomas Schmied, Fabian Paischer, Vihang Patil, Markus Hofmarcher, Razvan Pascanu, Sepp Hochreiter
• Human-Timescale Adaptation in an Open-Ended Task Space
  • Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw, Jack Parker-Holder, Shreya Pathak, Nicolas Perez-Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls, Sarah York, Alexander Zacherl, Lei Zhang
• Generalization to New Sequential Decision Making Tasks with In-Context Learning
  • Sharath Chandra Raparthy, Eric Hambro, Robert Kirk, Mikael Henaff, Roberta Raileanu
• Large Language Models can Implement Policy Iteration
  • Ethan Brooks, Logan Walls, Richard L. Lewis, Satinder Singh
• Learning How to Infer Partial MDPs for In-Context Adaptation and Exploration
  • Chentian Jiang, Nan Rosemary Ke, Hado van Hasselt
• Towards General-Purpose In-Context Learning Agents
  • Louis Kirsch, James Harrison, C. Daniel Freeman, Jascha Sohl-Dickstein, Jürgen Schmidhuber
• Large Language Models as General Pattern Machines
  • Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, Andy Zeng
• Cross-Episodic Curriculum for Transformer Agents
  • Lucy Xiaoyang Shi, Yunfan Jiang, Jake Grigsby, Linxi "Jim" Fan, Yuke Zhu
• First-Explore, then Exploit: Meta-Learning Intelligent Exploration
  • Ben Norman, Jeff Clune
• Meta-Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning
  • Tengye Xu, Zihao Li, Qinyuan Ren
• Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning
  • Jonathan Cook, Chris Lu, Edward Hughes, Joel Z. Leibo, Jakob Foerster
• Large Language Models As Evolution Strategies
  • Robert Tjarko Lange, Yingtao Tian, Yujin Tang
• Can Large Language Models Explore In-Context?
  • Akshay Krishnamurthy, Keegan Harris, Dylan J. Foster, Cyril Zhang, Aleksandrs Slivkins
• SAD: State-Action Distillation for In-Context Reinforcement Learning under Random Policies
  • Weiqin Chen, Santiago Paternain
• Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning
  • Subhojyoti Mukherjee, Josiah P. Hanna, Qiaomin Xie, Robert Nowak
• Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning
  • Jiuqi Wang, Ethan Blaser, Hadi Daneshmand, Shangtong Zhang