type annotations

[baggepinnen]

I have played around with and enjoyed this package and OpenAIGym.jl for a few days now. When trying to make my code run a bit faster, I checked for type instabilities etc. in my code, and I came to the conclusion that a lot of performance was lost due to the for (s,a,r,s') in ep iteration. The problem seems to be that the compiler doesn't know the types of s,a,r,s'. I mitigated the issue somewhat by (in my case) for (s::Vector{Float64}, a::Int, r::Float64, s1::Vector{Float64}) in ep, after which the compiler can optimize for the types I declared.

Maybe speed is not the goal with this package, but RL methods have a tendency to require some time, so it would be nice if things ran fast. Maybe this is something to keep in mind while continuing the development on Reinforce.jl

ep = Episode(env,policy)
@code_warntype next(ep,1)

Variables:
  #self#::Base.#next
  ep::Reinforce.Episode
  i::Int64
  env::Any
  s::Any
  A::Any
  r::Any
  a::Any
  last_reward::Any
  s′::Any
  #temp#@_11::Int64
  _::Int64
  #temp#@_13::Any
.
.
.