YiTan

on line 84 of prorportional.py ```python weights /= max(weights) # Normalize for stability ``` should be fixed since weights is a list ``` python weights = [ i/max(weights) for i...

dqn uses huberloss instead of mseloss

1

https://blog.openai.com/openai-baselines-dqn/ *... In the DQN Nature paper the authors write: “We also found it helpful to clip the error term from the update [...] to be between -1 and 1.“....

removed deprecated feature 'torch.autograd.Variable' from /samples/rainbow/lib/common and also changed the loss function from nn.MSELoss() -> nn.functional.mse_loss()

https://pytorch.org/docs/stable/distributions.html score functions and categorical sampling is already implemented in pytorch, using numpy should be discouraged. policy network should output a probability distribution

https://github.com/Shmuma/ptan/blob/84d349225f15a02164f28586b50cf94ee726eacc/ptan/experience.py#L497

https://github.com/Shmuma/ptan/blob/84d349225f15a02164f28586b50cf94ee726eacc/samples/rainbow/lib/common.py#L88