Crash when extending VecEnvWrapper

[GalAvineri]

🐛 Bug

When you try to extend the VecEnvWrapper, and put a breakpoint in the construction function of the new class, pycharm crashes due to some recursive call in VecEnv.

To Reproduce

You can reproduce this with the code given in the example to a vectorized environments wrapper found here

(I just replaced the environment to be cartpole and added imports to gym and DummyVecEnv)

import numpy as np

from stable_baselines3.common.vec_env.base_vec_env import VecEnv, VecEnvStepReturn, VecEnvWrapper
from stable_baselines3.common.vec_env.dummy_vec_env import DummyVecEnv
import gym


class VecExtractDictObs(VecEnvWrapper):
    """
    A vectorized wrapper for filtering a specific key from dictionary observations.
    Similar to Gym's FilterObservation wrapper:
        https://github.com/openai/gym/blob/master/gym/wrappers/filter_observation.py

    :param venv: The vectorized environment
    :param key: The key of the dictionary observation
    """

    def __init__(self, venv: VecEnv, key: str):
        self.key = key
        super().__init__(venv=venv, observation_space=venv.observation_space.spaces[self.key])

    def reset(self) -> np.ndarray:
        obs = self.venv.reset()
        return obs[self.key]

    def step_async(self, actions: np.ndarray) -> None:
        self.venv.step_async(actions)

    def step_wait(self) -> VecEnvStepReturn:
        obs, reward, done, info = self.venv.step_wait()
        return obs[self.key], reward, done, info

env = DummyVecEnv([lambda: gym.make("CartPole-v0")])
# Wrap the VecEnv
env = VecExtractDictObs(env, key="observation")

If you put a breakpoint inside the body of the __init__ function, for example on the self.key = key line, pycharm will crash with the following trace:

Current thread 0x000065ac (most recent call first):
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 303 in __getattr__
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 320 in _get_all_attributes
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 346 in getattr_depth_check
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 303 in __getattr__
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 320 in _get_all_attributes
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 346 in getattr_depth_check
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 303 in __getattr__
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 320 in _get_all_attributes
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 346 in getattr_depth_check
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 303 in __getattr__
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 320 in _get_all_attributes
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 346 in getattr_depth_check
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 303 in __getattr__
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 320 in _get_all_attributes
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 346 in getattr_depth_check
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 303 in __getattr__
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 320 in _get_all_attributes
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 346 in getattr_depth_check
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 303 in __getattr__
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 320 in _get_all_attributes
  File "C:\Users\gal.avineri\Miniconda3\envs\optimization\lib\site-packages\stable_baselines3\common\vec_env\base_vec_env.py", line 346 in getattr_depth_check

### System Info OS: Windows-10-10.0.19042-SP0 10.0.19042 Python: 3.9.7 Stable-Baselines3: 1.4.0 PyTorch: 1.10.2 GPU Enabled: False Numpy: 1.21.5 Gym: 0.19.0

stable-baselines3 was installed via pip. pycharm version is 2021.3.1 professional.

Checklist

• [x] I have checked that there is no similar issue in the repo (required)
• [x] I have read the documentation (required)
• [x] I have provided a minimal working example to reproduce the bug (required)

[araffin]

Hello, to reproduce your issue, you need to add import ipdb; ipdb.set_trace() before the call to super() and then enter self.this_attr_does_not_exist for instance in the command line. If you put your breakpoint after the call to super(), everything is fine.

Do you know what is pycharm doing internally when you put a breakpoint? (I'm always using the ipython debugger)

[araffin]

I just tested with pycharm with the provided code and could not reproduce the issue... (setting a breakpoint at self.key = key)

import numpy as np

from stable_baselines3.common.vec_env.base_vec_env import (
    VecEnv,
    VecEnvStepReturn,
    VecEnvWrapper,
)
from stable_baselines3.common.vec_env.dummy_vec_env import DummyVecEnv
import gym


class CustomEnv(gym.Env):
    def __init__(self):
        super(CustomEnv, self).__init__()
        self.observation_space = gym.spaces.Dict(
            {"observation": gym.spaces.Box(low=-np.inf, high=np.inf, shape=(14,))}
        )
        self.action_space = gym.spaces.Box(low=-1, high=1, shape=(6,))

    def reset(self):
        return self.observation_space.sample()

    def step(self, action):
        obs = self.observation_space.sample()
        reward = 1.0
        done = False
        info = {}
        return obs, reward, done, info


class VecExtractDictObs(VecEnvWrapper):
    """
    A vectorized wrapper for filtering a specific key from dictionary observations.
    Similar to Gym's FilterObservation wrapper:
        https://github.com/openai/gym/blob/master/gym/wrappers/filter_observation.py

    :param venv: The vectorized environment
    :param key: The key of the dictionary observation
    """

    def __init__(self, venv: VecEnv, key: str):
        self.key = key
        super().__init__(
            venv=venv, observation_space=venv.observation_space.spaces[self.key]
        )
        #import ipdb; ipdb.set_trace()


    def reset(self) -> np.ndarray:
        obs = self.venv.reset()
        return obs[self.key]

    def step_async(self, actions: np.ndarray) -> None:
        self.venv.step_async(actions)

    def step_wait(self) -> VecEnvStepReturn:
        obs, reward, done, info = self.venv.step_wait()
        return obs[self.key], reward, done, info


env = DummyVecEnv([lambda: CustomEnv()])
# Wrap the VecEnv
env = VecExtractDictObs(env, key="observation")

Pycharm version: PyCharmCE2021.3 Python 3.7 on Ubuntu

OS: Linux-5.4.0-105-generic-x86_64-with-debian-bullseye-sid #119-Ubuntu SMP Mon Mar 7 18:49:24 UTC 2022 Python: 3.7.9 Stable-Baselines3: 1.5.1a0 PyTorch: 1.11.0+cpu GPU Enabled: False Numpy: 1.20.2 Gym: 0.21.0

[GalAvineri]

I ran the code you provided, making a few specs more similar to yours (gym, pycharm and sbs3 versions) and it still crashed.

OS: Windows-10-10.0.19044-SP0 10.0.19044 Python: 3.9.5 Stable-Baselines3: 1.5.0 PyTorch: 1.11.0+cpu GPU Enabled: False Numpy: 1.22.3 Gym: 0.21.0

I noticed that there are 3 differences between our specs that may be relevant: The os (windows vs linux), sb3 version and python version.