JuliaReinforcementLearning
Space concept
We need to decide on a concept for spaces (e.g. the action space and observation space). One option would be to have an AbstractSpace type. I am against this. Instead, I think we should rely on duck typing, for example, spaces might support in, rand, eltype, iterate. In particular, I strongly suggest that [1,2,3,4] should be a valid action space.
Continuous spaces are a bit more difficult, but I think this would best be handled by a separate package called BoxSets or something. See also JuliaPOMDP/POMDPs.jl#301
Yes, I would avoid introducing a type hierarchy unless it provides something via dispatch (usually performance). I think a good alternative to ensure consistency across implemented environments is to write out best practices. Many spaces have special names, but AFAIK that's a consequence of software not literature. On the other hand, many spaces are extremely generic concepts that are not tied to RL at all. I suggest we put some effort into collecting existing Julia implementations of these generic spaces. Most might just be in Base, but I think some might be from other packages (e.g. interval sets).
Yeah that's true. Maybe making a PR to add Cartesian products + rand would be either than writing the whole thing from scratch. Either way out of the scope of this package in my opinion.
Ok, I think we agree that there should not be a type hierarchy, but we will need to answer some questions. The first is:
As an algorithm-writer, how do I tell if the set is discrete or continuous? (maybe something like IteratorSize?)