rl

rl copied to clipboard

Document the TensorDict structure of the return of the _step() function for a multi agent environment

2

## Motivation It is not very clear what should be the structure of TensorDict of the return of the _step() function for a multi agent environment. If there are two...

zoetsekas

[Doc] Better doc for distributed RBs

3

vmoens

[BUG] remove error catches (try/except) in objectives

2

We still have a bunch of try/except in losses such as PPO to compute the entropy. We need to remove them for compile compatibility.

vmoens

[BUG] sota-implementation requires nightly built

2

## Describe the bug dqn_cartpole from sota-implementations/dqn doesn't working. Crashes with: **ImportError: cannot import name 'Composite' from 'torchrl.data'** ## To Reproduce Just run dqn_cartpole.py, it will call ulits_cartpole.py and then...

jimdor

[Feature] use_vmap=False for SAC

3

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #2393 * __->__ #2392

vmoens

[Feature] non-functional SAC loss

3

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #2393 * #2392

vmoens

## Motivation Attempting to implement [Parallel Q Networks](https://www.researchgate.net/publication/382080747_Simplifying_Deep_Temporal_Difference_Learning) (online DQN without replay buffer or target networks). Uses QLambda returns. ## Solution TDLambdaEstimator expects `state_value` keys but we would now need...

roger-creus

[Feature] flexible batch_locked for jumanji

3

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #2382

vmoens

[Algorithm] TD3 fast

3

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #2389

vmoens

[Feature Request] Metadata for specs

2

> Hi there. Besides the naming, what do you think of adding some metadata for users to populate? > > This could be useful for, for example, marking which keys...

vmoens