Gym examples don't represent ideal solutions

[NivenT]

I guess first off, ideally there would be an example for every training algorithm added. For now, the best way I've found to set these up seems to be to use OpenAI's gym as a source of environments.

Unfortunately though, this library really only offers tables and linear approximations as possible value functions. These are not always ideal for gym environments, so getting working examples requires appropriate choices of features and hyperparameters in order for the agent to learn well. However, doubly unfortunately, I'm still new to RL so the current solutions are not ideal, and can likely be improved.

For some context, you can

• compare the performance of the cartpole agent before and after this commit: 47e8d161e58277f84cbef7ba12c50add5199620d
• see how well the mountain car agent does before it was removed in this commit: d3beac938e42d2f454ae0e944af5932238a89347

To sum it up, the issue here is the example solutions should be designed more carefully.