Paper Collection of Recommender System

This is a collection of research and review papers of reinforcement learning (RL) based recommender system techniques. Several learning-to-rank (LTR) research papers are also listed here due to high correlation. All the papers are sorted by time. Any suggestions and pull requests are more than welcome.

The sharing principle of these references here is for research. If any authors do not want their paper to be listed here, please feel free to contact Jiarui Jin (Email: jinjiarui97 [AT] gmail.com).

Review Papers

• Reinforcement Learning for Online Information Seeking by Xiangyu Zhao, Long Xia, Jiliang Tang, Dawei Yin. arXiv, 2019.

Research Papers

• Validation Set Evaluation can be Wrong: An Evaluator-Generator Approach for Maximizing Online Performance of Ranking in E-commerce by Guangda Huzhang, Zhen-Jia Pang, Yongqing Gao, Wen-Ji Zhou, Qing Da, An-Xiang Zeng, Yang Yu. arXiv, 2020.
• Hierarchical Adaptive Contextual Bandits for Resource Constraint based Recommendation by Mengyue Yang, Qingyang Li, Zhiwei Qin, Jieping Ye. WWW, 2020.
• Deep Reinforcement Learning for Whole-Chain Recommendations by Xiangyu Zhao, Long Xia, Dawei Yin, Jiliang Tang. WSDM, 2020.
• MBCAL: A Simple and Efficient Reinforcement Learning Method for Recommendation Systems by Fan Wang, Xiaomin Fang, Lihang Liu, Hao Tian, Zhiming Peng. arXiv, 2019.
• MarlRank: Multi-agent Reinforced Learning to Rank by Shihao Zou, Zhonghua Li, Mohammad Akbari, Jun Wang, Peng Zhang. CIKM, 2019.
• Deep Learning Recommendation Model for Personalization and Recommendation Systems by Maxim Naumov, Dheevatsa Mudigere, Hao Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong and Misha Smelyanskiy. arXiv, 2019.
• Reducing Exploration of Dying Arms in Mortal Bandits by Stefano Tracà, Cynthia Rudin, Weiyu Yan. arXiv, 2019.
• Bandit Learning for Diversified Interactive Recommendation by Yong Liu, Yingtai Xiao, Qiong Wu, Chunyan Miao, Juyong Zhang. arXiv, 2019.
• Explainable Knowledge Graph-based Recommendation via Deep Reinforcement Learning by Weiping Song, Zhijian Duan, Ziqing Yang, Hao Zhu, Ming Zhang, Jian Tang. arXiv, 2019.
• Deep Reinforcement Learning for Online Advertising in Recommender Systems by Xiangyu Zhao, Changsheng Gu, Haoshenglun Zhang, Xiaobing Liu, Xiwang Yang, Jiliang Tang. arXiv, 2019.
• Value-aware Recommendation based on Reinforced Profit Maximization in E-commerce Systems by Changhua Pei, Xinru Yang, Qing Cui, Xiao Lin, Fei Sun, Peng Jiang, Wenwu Ou, Yongfeng Zhang. WWW, 2019.
• Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recommender Systems by Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Peng He, Paul Weng, Han Gao, Guihai Chen. WWW, 2019.
• Reinforcement Knowledge Graph Reasoning for Explainable Recommendation by Yikun Xian, Zuohui Fu, S. Muthukrishnan, Gerard de Melo, Yongfeng Zhang. SIGIR, 2019.
• Variance Reduction in Gradient Exploration for Online Learning to Rank by Huazheng Wang, Sonwoo Kim, Eric McCord-Snook, Qingyun Wu, Hongning Wang. SIGIR, 2019.
• Mention Recommendation in Twitter with Cooperative Multi-Agent Reinforcement Learning by Tao Gui, Peng Liu, Qi Zhang, Liang Zhu, Minlong Peng, Yunhua Zhou, Xuanjing Huang. SIGIR, 2019.
• SlateQ: A Tractable Decomposition for Reinforcement Learning with Recommendation Sets by Eugene Ie, Vihan Jain, Jing Wang, Sanmit Narvekar, Ritesh Agarwal, Rui Wu, Heng-Tze Cheng, Tushar Chandra, Craig Boutilier. IJCAI, 2019.
• Generative Adversarial User Model for Reinforcement Learning Based Recommendation System by Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, Le Song. ICML, 2019.
• Large-scale Interactive Recommendation with Tree-structured Policy Gradient by Haokun Chen, Xinyi Dai, Han Cai, Weinan Zhang, Xuejian Wang, Ruiming Tang, Yuzhou Zhang, Yong Yu. AAAI, 2019.
• Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems by Lixin Zou, Long Xia, Zhuoye Ding, Jiaxing Song, Weidong Liu, Dawei Yin. KDD, 2019.
• Top-K Off-Policy Correction for a REINFORCE Recommender System by Minmin Chen, Alex Beutel, Paul Covington, Sagar Jain, Francois Belletti, Ed H. Chi. WSDM, 2019.
• Environment Reconstruction with Hidden Confounders for Reinforcement Learning based Recommendation by Wenjie Shang, Yang Yu, Qingyang Li, Zhiwei Qin, Yiping Meng, Jieping Ye. KDD, 2019.
• Deep Reinforcement Learning for List-wise Recommendations by Xiangyu Zhao, Liang Zhang, Long Xia, Zhuoye Ding, Dawei Yin, Jiliang Tang. DRL4KDD, 2019.
• Purchase as Reward : Session-based Recommendation by Imagination Reconstruction by Qibing Li, Xiaolin Zheng. Open review at ICLR 2019.
• Reinforcement Learning to Rank in e-commerce search engine: Formalization, analysis, and application by Yujing Hu, Qing Da, Anxiang Zeng, Yang Yu, Yinghui Xu. KDD, 2018.
• Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning by Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, Dawei Yin. KDD, 2018.
• Stabilizing Reinforcement Learning in Dynamic Environment with Application to Online Recommendation by Shi-Yong Chen, Yang Yu, Qing Da, Jun Tan, Hai-Kuan Huang, Hai-Hong Tang. KDD, 2018.
• Optimizing Query Evaluations using Reinforcement Learning for Web Search by Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary. SIGIR, 2018.
• A Reinforcement Learning Framework for Explainable Recommendation by Xiting Wang, Yiru Chen, Jie Yang, Le Wu, Zhengtao Wu, Xing Xie. ICDM, 2018.
• Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learning by Jing-Cheng Shi1, Yang Yu, Qing Da, Shi-Yong Chen, An-Xiang Zeng. AAAI, 2018.
• Deep Reinforcement Learning for Page-wise Recommendations by Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, Jiliang Tang. RecSys, 2018.
• DRN: A Deep Reinforcement Learning Framework for News Recommendation by Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, Zhenhui Li. WWW, 2018.
• Learning to Collaborate: Multi-Scenario Ranking via Multi-Agent Reinforcement Learning by Jun Feng, Heng Li, Minlie Huang, Shichen Liu, Wenwu Ou, Zhirong Wang, Xiaoyan Zhu. WWW, 2018.
• Reinforcement Learning to Rank with Markov Decision Process by Zeng Wei, Jun Xu, Yanyan Lan, Jiafeng Guo, Xueqi Cheng. SIGIR, 2017.
• Deep Reinforcement Learning in Large Discrete Action Spaces by Gabriel Dulac-Arnold, Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, TheophaneWeber, Thomas Degris, Ben Coppin. arXiv, 2016.
• Personalized Ad Recommendation Systems for Life-Time Value Optimization with Guarantees by Georgios Theocharous, Philip S. Thomas, Mohammad Ghavamzadeh. IJCAI, 2015.
• Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms by Lihong Li, Wei Chu, John Langford, Xuanhui Wang. WSDM, 2011.