Sharad Chitlangia

RPC Communication in Distributed RL Training

2

There's three ways that I can think of having distributed training: 1. Use of Pytorch's Distributed Training infrastructure. Would require establishing communication protocols specific to the case of Deep RL....

Logger Formatting

12

The current logger might go on to the next line if there are a lot of key, value pairs. There could be three solutions to this: 1. Put a limit...

Distributional Agents

1

Agents should be structured in a way that they can be extended to distributional or distributed agents (and both as well, case in point: D4PG and lots of others :))....

save_to_gif argument in Trainer

3

Save a GIF file based on this argument in trainer. To-do: 1. Check tensorboard saving in video

Custom Loss Functions for RL

3

We should think about common loss functions that are used a lot in RL that can be packaged. As of now, we're constructing everything from scratch so we're going towards...

Mujoco, PyBullet support

12

We should develop an environment module with wrappers. For a starter, I find [TF Agents env module](https://github.com/tensorflow/agents/tree/master/tf_agents/environments) pretty good.

Off Policy Algos for CBs

1

1. DoublyRobustEstimator 2. Etc

Trainer and Agent envs object

1

Objects should be the same else an error should be raised.

Evaluation Methods for CBs

1

1. Doubly-Robust Policy Evaluation 2. Etc

Allow Logger free form printing

3

Current Logging only allows printing dictionaries, a free form should be allowed too. Some thinking needs to be done as to how to make this happen without breaking the regular...