microsoft
Transition to Gymnasium / Future Compatibility with Tianshou
The maintainers of OpenAI Gym recently forked Gym to Gymnasium, which is where all future maintenance will occur. More information can be found in the announcement blog post of the Farama Foundation.
Tianshou will, along with many other large RL frameworks, transition towards Gymnasium. We are currently working on a PR for this: https://github.com/thu-ml/tianshou/pull/789. In particular, Tianshou will stop supporting bare Gym environments, especially those that follow the old API.
NNI depends on Tianshou via PolicyBasedRL. The transition of Tianshou to Gymnasium would break this since ModelEvaluationEnv is an old-style Gym environment. This can easily be fixed by updating the environment to Gymnasium. Gymnasium is currently a drop-in replacement for OpenAI Gym, so this shouldn't be too hard. However, the environment should conform with the new API (already introduced in OpenAI Gym 0.26), which would require some simple changes, as explained here.
I'd be happy to help with any questions or PRs. @jkterry1 @Trinkle23897
There is coming new implementation of PolicyBasedRL in NNI v3.0. The large part of coding work is done and I just incorporate the new API of Gymnasium: https://github.com/microsoft/nni/commit/2efe0c6502d354d99181e05256823063458854ed
It will come into the master branch of NNI shortly. If tianshou has encountered any problems in CI, feel free to disable them temporarily. I'll address those problems after v3.0 is released.
Thanks for the notification.
What versions of Tianshou, Gym and shimmy could be used in a NNI v2.7 environment in order to work with RL?
