[RLlib] Support terminated and truncated in ExternalMultiAgentEnv

[Paolo-Fermin]

Why are these changes needed?

The ExternalMultiAgentEnv class accepts a multiagent_done_dict as a part of the log_returns function. However, given the recent change to the gym API to gymnasium, dones have been split into terminated and truncated booleans. This PR updates the ExternalMultiAgentEnv api to accept multiagent_terminated_dict as well as multiagent_truncated_dict, while retaining the old multiagent_done_dict for backwards compability. In the case that the user provides the deprecated multiagent_done_dict, assume that they want to set both terminated and truncated simultaneously.

Related issue number

Closes #35189

Checks

• [x] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• [x] I've run scripts/format.sh to lint the changes in this PR.
• [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
  • [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file.
• [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • [x] Unit tests
  • [ ] Release tests
  • [ ] This PR is not tested :(

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[dogganon]

It's been a year and THIS PR STILL HAS NOT BEEN MERGED??

@ericl @sven1977 That being said, y'all NEED TO SERIOUSLY take a look at the related https://github.com/ray-project/ray/issues/35189 to see how severe the bug IS!! AND MERGE THIS PR ASAP!!!! multiagent_done_dict in ray.rllib.env.ExternalMultiAgentEnv.log_returns IS BASICALLY UNUSABLE DUE TO ray.rllib.env.external_env._ExternalEnvEpisode.cur_done_dict BEING NOT DEFINED! THIS COULD HAVE BEEN EASILY PREVENTED IF THERE WERE AT LEAST SOME UNIT TESTS IN PLACE! MIND BLOWING!!

SO TLDR, MERGE THIS PR ASAP AND ADD UNIT TESTS FOR ray.rllib.env.ExternalMultiAgentEnv!! 🤯

@sven1977 @gjoliver @avnishn @ArturNiederfahrenhorst @smorad @maxpumperla @kouroshHakha @krfricke

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[github-actions[bot]]

This pull request has been automatically closed because there has been no more activity in the 14 days since being marked stale.

Please feel free to reopen or open a new pull request if you'd still like this to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for your contribution!