Quinn Sinclair

N-step updates for off-policy methods

37

## Changes #### OffPolicyAlgorithm - Now accepts buffer class as argument type - Rationale: allows for other types of replay buffers besides the current implementations - Now accepts arguments for...

[Optimization] Replay Buffers shouldn't use copy when using np.array

10

Related to #49 In https://github.com/DLR-RM/stable-baselines3/blob/23afedb254d06cae97064ca2aaba94b811d5c793/stable_baselines3/common/buffers.py#L198-L208 https://github.com/DLR-RM/stable-baselines3/blob/23afedb254d06cae97064ca2aaba94b811d5c793/stable_baselines3/common/buffers.py#L346-L349 We call `np.ndarray(x).copy()`. This is unnecessary because np.array has the argument "copy" which is True by default. https://numpy.org/doc/stable/reference/generated/numpy.array.html ```python import numpy as np x...

[Proposal, Enhancement] Improve Registry through class initialisation arguments

6

# Proposal Using class initialization arguments allow for very easy class registration, this feature makes registering, using, and retrieving policies through strings trivial. As is, every algorithm provides a `BasePolicy`...

[Feature-request] Allow auxiliary tasks in policies through a unified interface

1

## Motivation There is a large corpus of work that shows that auxiliary tasks (i.e. anything besides learning a policy and a value function) can help agents get better state...

[feature-request] Use torch's jit to reduce walltime

3

## Motivation RL, unlike DL, is reliant on both CPU and GPU performance, simply because we interact with the environment. DL is not reliant on the CPU because the processing...

Is it intended for this theme to show both up and start with the same icon?

1

I was running OT, initialized from python, with random theme, random seed and here the door to the next floor is the same as the starting door. Is this intended?

Unsolvable floors

5

i encountered an unsolvable floor, the tower is 75 and the floor is 20. There is a box puzzle and key requirement. There is no tile to place the box...

[Bug] DQN Exploration divides by 0 when learn steps are small

10

Training a DQN agent for a few steps fails because of a divide by zero bug here: ```python class LinearSchedule(Schedule): # # .... other code .... # def value(self, step):...

## Lapce Version 0.2.5 ## System information Macbook pro 2021: MacOS 12.6.1, ## Describe the bug When 7 or more files / tabs are open, clicking on the last tab...

Windows Support

1

It's mainly the node build script that is relying on unix paths at the moment. Path.join is platform specific (which is strange since windows supports forward slashes since forever)