Consider splitting up agent and environment in separate containers

[david-woelfle]

Problem

Currently the agent and the environment are executed in the same container. This makes installing dependencies for new research algorithms (for agents) potentially complicated, especially as some of the environments seem to be quite old and it is likely only a matter of time before dependencies of the agent and the environment collide.

Potential Solution

IMHO it would be much better to split up the agent and environment into two containers. The benefits are:

• Dependencies of agent and environment can't collide.
• It would open a clean way how users can install packages required by their agent scripts.
• The size of the environment containers would shrink by likely ~80% as those containers do not need tensorflow etc.
• It would open the possibility for users to interact with the environments interactively like from inside a jupyter notebook.
• As the agent side of beobench would rather be really simple and without much dependencies (apart from the agent itself) it might even allow users to run the agent on their local system, which might be favorable for all those how are not too firm in docker.

I would be willing to support this development and could provide a minimal example for such a setup with celery if it would help you get started.