Add support for heterogeneous observations
-
Add the ability to mix non-homogeneous observations, for example images and with proprioceptive states.
-
Add example task: Cartpole balancing towards a red sphere. Observations: cartpole joint state, RGB image of the scene
Proposal (after some discussions with @kellyguo11): Define the spaces using both: gym.spaces and python basic data types (for simplicity) indistinctly in env configs files as follows. Use the names observation_space, action_space and state_space rather than num_observations, num_actions and num_spaces in configs files to define them respectively.
| gym.spaces | python basic data type |
|---|---|
Box |
int or list of int (e.g.: 10, [64, 64, 3]) |
Discrete |
set (e.g.: {2}) |
MultiDiscrete |
list of sets (e.g.: [{2}, {5}]) |
| gym.spaces | python basic data type |
|---|---|
Dict |
dict (e.g.: {"joints": 10, "camera": [64, 64, 3], "gripper": {2}}) |
Tuple |
tuple (e.g.: (10, [64, 64, 3], {2})) |
I am not a huge fan of having to manually specify this space since users (in robotics at least) are usually changing the observations and actions on the fly while prototyping the environment. Changing the observation function and then the gym.Space for it seems cumbersome and requires "counting" dimensions (which is very erroneous).
Particuarly, for the manager based environment workflow, we expect users to only provide the "terms" and we infer everything from those terms. Having to specify the numbers for explicit spaces would add more steps for the user (unless we can think of a nicer solution).
But in general, I agree. We should handle spaces somewhat correctly.