rl
rl copied to clipboard
pytorch
A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
## Motivation In the current implementation of the VMAS environment, the done fields are only available on the root of the tensordict. However, for training, it is useful to have...
## Motivation Compiling a back-to-back call to `ReplayBuffer.extend` and `ReplayBuffer.sample`, and then calling it multiple times causes the function to be recompiled each time. ```python import torch import torchrl torch._logging.set_logs(recompiles=True)...
## Describe the bug When run the example https://github.com/pytorch/rl/blob/main/examples/distributed/collectors/multi_nodes/ray_train.py i meet following bug: UserWarning: init_random_frames (-1) is not exactly a multiple of frames_per_batch (1000), this results in more init_random_frames than...
## Describe the bug The device of `info['_weight']` doesn't match the storage device. ## To Reproduce ```python # From documentation from torchrl.data.replay_buffers import ReplayBuffer, LazyTensorStorage, PrioritizedSampler from tensordict import TensorDict...
## Bug Description 1. During the initialization of `LossModule` and execution of `convert_to_functional()`, the first layer parameters are set as `UninitializedParameter(shape=torch.Size(-1))`. Consequently, when `target_value_network_params` is cloned from this, it becomes...
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #2507
## Motivation The `LossBase` class `make_value_estimator` method takes an Enum type `ValueEstimators` to construct the value estimator. Also, its docstring indicates that: > If the non-default value function is wanted,...
## Describe the bug The cell ``` from torchrl.envs import [GymEnv](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module) env = [GymEnv](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module)("CartPole-v1", from_pixels=True, pixels_only=False) print(env.rollout(max_steps=3)) from torchrl.envs import [TransformedEnv] ``` from tutorial # 4 (https://pytorch.org/rl/stable/tutorials/getting-started-4.html) returns `ModuleNotFoundError: No...
## Describe the bug When I proceeded further based on the issue https://github.com/pytorch/rl/issues/2402 I have already made modifications based on the suggestions according to the issue above, but my code...
## Describe the bug When I proceeded further based on the issue [https://github.com/pytorch/rl/issues/2402](https://github.com/pytorch/rl/issues/2402), I encountered this warning. I have pulled the GitHub projects **torchrl** and **tensordict**, and installed the latest...