[Feature Request] Integration for OpenSpiel environments

[Brunozml]

Motivation

It would be great to be able to use OpenSpiel's environments with TorchRL.

Solution

Its rl_environment interface is basically identical to that of gym, so the integration should relatively straightforward. I'm more than willing to help achieve this.

Alternatives

I'm not sure if it's already possible to do this using GymLikeEnv, but so far my experimentation has failed.

Additional context

Checklist

• [X] I have checked that there is no similar issue in the repo (required)

[vmoens]

That would be wonderful!

Never tried it to be completely open about that.

Looking at it the API seems quite different from the one from gym.

IIUC, the steps to achieve this would be:

• map actions to tensors, map states to tensors
• find a generic way to generate specs, check that actions and states mapped above are within the specs, and that specs generate valid states and action at random
• I guess the env will be stateless so we will need a copy of the state specs in state_spec and observation_spec

Concretely, I'd love to see a dummy rollout where we get the state (represented as a single array or multiple arrays or tensors) and the action (idem).

If the games have anything remotely related to "specs" i'd like to see them too!

[kurtamohler]

I'm looking into the possibility of making the OpenSpiel wrapper stateless. I'm not sure yet if it's possible to obtain a dict-of-arrays representation of an OpenSpiel state. I've submitted an issue in OpenSpiel asking about it: https://github.com/google-deepmind/open_spiel/issues/1254

[kurtamohler]

I would like to reopen this, since I still have to add support for chance nodes