Max Weissenbacher

[BUG] VecNorm.to_observation_norm broken for multiple keys

1

## Describe the bug The VecNorm transform produces a bug when calling the to_observation_norm method with multiple keys ## To Reproduce ``` gym_env = GymEnv("MountainCarContinuous-v0", device='cpu') transformed_env = TransformedEnv(gym_env, VecNorm(in_keys=["observation",...

[Feature Request] Add a Transform that remembers the previous action taken

4

## Motivation I would propose implementing a simple Transform that remembers the previous action taken. This can be very useful when considering environments with time delay, or when training agents...

[Algorithm] Added TQC

10

## Description Added implementation of TQC algorithm. I've tried to stay as close to the original PyTorch implementation as possible. - Implementation is based on the implementation of SAC in...

[Feature Request] FrameSkipTransform - output every intermediate step?

6

## Motivation A feature request regarding the `FrameSkipTransform` ([documentation here](https://pytorch.org/rl/reference/generated/torchrl.envs.transforms.FrameSkipTransform.html#torchrl.envs.transforms.FrameSkipTransform)). When the `frame_skip` parameter is set to an integer greater than 1, i.e. `frame_skip = 2`, and I do a...

[Feature Request] Add linear interpolation to FrameSkipTransform

3

## Motivation I would suggest adding an option for linear interpolation to the `FrameSkipTransform` ([documentation here](https://pytorch.org/rl/reference/generated/torchrl.envs.transforms.FrameSkipTransform.html#torchrl.envs.transforms.FrameSkipTransform)). This means, instead of repeating the same action `frame_skip` number of times for each...

[Feature Request] Implement TQC for the example algorithms

6

## Motivation I suggest adding an implementation of [TQC](https://github.com/SamsungLabs/tqc_pytorch) to the examples. I suggest adding this as a request in the 'call for distributions' stack. I would be happy to...

Incompatible with PyTorch 2.0 - variable modified inplace

1

Getting the following error when trying to run the code with a (very simple) custom env using PyTorch 2.0.1: `RuntimeError: one of the variables needed for gradient computation has been...

Okay this might be a very silly question. In all the things that get logged - _where is the evaluation reward_? Can someone please tell me?

Incompatibility with latest firedrake version

5

Hi there, simple thing: just installed the latest version of Firedrake. When trying to import `import hydrogym.firedrake as hgym` I get the following error: `ImportError: cannot import name 'atan_2' from...

The demo in Quickstart does not work

1

Just wanted to let you know that the demo you provide in the Quickstart documentation does not work. Getting this error: ``` self.solver: TransientSolver = env_config.get("solver")( ^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object...