[DO NOT CLOSE] Call for contributions

[vmoens]

This issue is a list of contributions requests from the community.

How to use this list

If you're willing to contribute to the library, have a look at the list below and don't hesitate to pick up a task. If you need guidance, refer to @vmoens for more information! Once you pick up a task, assign the related issue to yourself, to make sure that no other collaborator is working on the same task at the same time (or create an issue if there isn't one already).

If you want to add an item to this list, start by raising an issue and mention that you think it would be appropriate to put it in the "call for contributions" stack.

If it's your first contribution, warm up with an issue marked with https://github.com/github/docs/labels/Good%20first%20issue label (and claim the issue so no one else does!)

New algorithms

New algorithms can be coded either in a free form or using torchrl's trainer class. In any case, we would ask to the user to use hydra for the configuration, and to limit the number of extra dependencies as much as can be.

• [ ] PILCO (no open issue yet)
• [ ] Image augmentation is all you need #32
• [ ] TQC algorithm (https://github.com/pytorch/rl/issues/1623)
• [ ] A3C #1755

## New environment libraries

• [ ] Safety gym compatibility
• [ ] Procgen
• [ ] Genesis

New modules and features

• [X] Reward-to-go #16
• [ ] On-the-fly adaptation of alpha and beta in PRB #1575
• [ ] Raise exception when sampling from empty replay buffer (#994)
• [ ] Add an option to "squash" the observation dictionary in register_gym when there is only one observation (ie, not return a dict but a simple tensor)
• [ ] Add a num_envs option in DMControlEnv to create a parallel env in just one call, e.g. DMControlEnv(name, task, num_envs=4) would run 4 parallel envs of the name, task dmc env.

Datasets

• [ ] MIMIC #1679
• [ ] SG2EGSet
• [ ] TDMPC2
• [ ] DexGraspNet A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation
• [ ] Ego4d
• [ ] BridgeDataV2
• [ ] RoboVQA
• [ ] RL Unplugged
• [ ] d3rlpy

BugFixes

• [ ] Colabs don't render in the doc
• [ ] Raise an meaningful exception when one-hot specs are reshaped with a shape that doesn't match the last dim for all transforms that incur a change of shape #1904

Deprecation calls

• [ ] Softly deprecate NormalParamWrapper in favor of NormalParamExtractor
• [ ] Deprecate wrappers https://github.com/pytorch/rl/blob/69d44f5cf4bf84eab0f21b0eea98112651f7f9a1/torchrl/modules/tensordict_module/exploration.py#L252 and https://github.com/pytorch/rl/blob/69d44f5cf4bf84eab0f21b0eea98112651f7f9a1/torchrl/modules/tensordict_module/exploration.py#L385 in favor of simple modules like https://github.com/pytorch/rl/blob/69d44f5cf4bf84eab0f21b0eea98112651f7f9a1/torchrl/modules/tensordict_module/exploration.py#L31

Solved issues

• [x] SMAC (Starcraft Multi-agent challenge) -> #810
• [x] PettingZoo
• [x] A2C algorithm #17 -> solved as of #702
• [x] TD3 algorithm #18 -> #684
• [x] Decision transformers #15
• [x] DQN Atari (https://offline-rl.github.io/). A D4RL wrapper can already be found here, which can be a good source of inspiration. #1815
• [x] Gen DGRL (WebShop and ProcGen) => #1678
• [x] Roboset #1743
• [x] V-D4RL (issue #1674) => #1756
• [x] Open X-Embodiment: Robotic Learning Datasets and RT-X Models #1751
• [x] Minari #1721

Thanks for contributing to TorchRL!