Cropping won't work for all environments

[claymcleod]

Hey @sherjilozair/everyone,

Just letting you know, this almost certainly won't work for any arbitrary game — at least, correctly — in its current form. The problem is the cropping: cropping a static 84x84 window in the center will miss a lot of important information.

Take for instance SpaceInvaders-v0. This is what the convolutional network sees: . Where is the ship that we control? It is cropped off the bottom of the screen. Thus, this information can't be contained within the network.

I appreciate that this is just a simple implementation, but it's worth noting that a complete solution would have hand picked regions over which to crop for each game (unfortunately).