How to render performance of agent visually

[Chazzz]

It would be nice to show how the agent is performing inside the game, but in the meantime here's some imperfect but functional code.

Note: This dramatically slows the throughput of the model. Only render after the model has finished training. May require additional dependencies (like matplotlib) if not already present.

Inside dopamine/atari/run_experiment.py:

  def _run_one_step(self, action):
    """Executes a single step in the environment.

    Args:
      action: int, the action to perform in the environment.

    Returns:
      The observation, reward, and is_terminal values returned from the
        environment.
    """
    self._environment.render(mode='human') #Shows game in progress. Note: Slows down training tremendously.
    time.sleep(1/60)
    observation, reward, is_terminal, _ = self._environment.step(action)
    return observation, reward, is_terminal

[sturdyplum]

I think there is a simpler way to do this (without having to edit dopamine source code). You could simply wrap your environment with https://github.com/google/dopamine/blob/76cdae1f858233a8501e2b61095cde54c6f8a214/dopamine/discrete_domains/gym_lib.py#L306

Then you can put the render code in the step function.

[psc-g]

hey everyone, just pushed some utilities to allow you to render the agent interacting with the environment (as well as other plots alongside it). check out this colab for an example on how to use it: https://colab.research.google.com/github/google/dopamine/blob/master/dopamine/colab/agent_visualizer.ipynb