Changes in DEP-RL library after myosuite Update

[YusseffRuiz]

After the last myosuite Update, the Tutorials for training a new policy for MyoSuite locomotion in MyoLeg, and i'm thinking in all other environments, there are a few changes needed to be performed so it can work fine:

Reproduce the Issue: Install Myosuite. Prepare Tutorial based on the following: https://myosuite.readthedocs.io/en/latest/tutorials.html "DEP-RL baselines" And Training file with the following: https://deprl.readthedocs.io/en/latest/loading_checkpoints.html#continuing-training-with-deprl

A new environment is required to be registered in init file in site-packages/deprl/

Afterwards, two error pops out:

Expected float and received Double.

Which can be solved by modifying every torch and arrays of the type float32, and changing them to "double" in the following files inside: deprl/custom_distributed

Then, an issue with compatibility with the new gym update appears, since "step" and "reset" returns 5 values, instead of 4.

We need to change the following in the file: deprl/env_wrappers/wrappers

From:

     def step(self, action):
        try:
            observation, reward, done, info= self._inner_step(action)
...

    def reset(self, **kwargs):
        observation = super().reset(**kwargs)

To

    def step(self, action):
        try:
            observation, reward, done, info, extras = self._inner_step(action)

       def reset(self, **kwargs):
        observation, *_ = super().reset(**kwargs)

After these changes, the repository works correctly with different environments.