Query about Integration of Transformer Models

[antonio-guillenperez]

Hi,

I am interested in your work on heterogeneous-agent reinforcement learning and the algorithms you have developed. I noticed that your current implementation focuses on algorithms like HAPPO, HATRPO, etc., based on PyTorch. Given the success of transformer models in various domains, are there any plans to incorporate transformer neural networks into your HARL algorithms? This could potentially enhance the capability of handling more complex heterogeneous agent interactions.

Thank you for considering my query.

[guazimao]

Hi. Your idea is very interesting. Combining transformer with HARL algorithms indeed has the potential to further improve performance. Your idea is similar to the paper "Multi-Agent Reinforcement Learning is a Sequence Modeling Problem", which uses the transformer, sequential update scheme, and PPO update objectives. You can take a look at this paper and its repo. If you have any further questions, feel free to discuss with me.