ray
ray copied to clipboard
[RLlib] - `MultiAgentEnvRunner` misses multi-agent metrics
What happened + What you expected to happen
What happened
I ran the tuned example [multi_agent_pendulum_ppo_envrunner.py](https://github.com/ray-project/ray/blob/master/rllib/tuned_examples/ppo/multi_agent_pendulum_ppo_envrunner.py)
and did not see single agent metrics in Tensorboard. Debugging showed, they are not collected.
What you expected to happen
That the new EnvRunner API
collects single agent metrics in a multi-agent setting like the new Learner API
does.
Versions / Dependencies
Ray nightly (Feb. 17th) Linux Fedora 37 Python 3.11.0
Reproduction script
multi_agent_pendulum_ppo_envrunner.py
Issue Severity
Medium: It is a significant difficulty but I can work around it.