how to wrap two DIFFERENT environments?

[superfan123]

Thank you for upload the package. My question is is there a way to stack different environments together? For example I have ten or hundreds different race track environments and I want to train an agent simultaneously drive through this vectorized environment. In stable baseline I can stack them together and train a vectorized environment. Now I want to move to ray and try to speed up the training by using multiple gpu...but so far didn't figure out how to do this. Thanks in advance

[ingambe]

Sorry for the late reply. As of today, this is not possible to do. However, it is possible to modify the RayEnvWrapper to do such a thing by passing a list of make_env functions to the VecEnv (like stable-baselines). I will keep this issue open once I have some free time to open a PR.

Edit: The only limit I can see would be to have similar observation space for the different environments, as we need to provide outputs of the same shape

[eric000888]

I think that's not a limitation as all the env in VecEnv also has same shape for observation space and action space to function properly in the vectorized setting. Ray 2.2 also introduced multi-env concept and I think that's can be done. Not sure how long it takes.