POMDPs.jl icon indicating copy to clipboard operation
POMDPs.jl copied to clipboard

We need some convenient space types

Open zsunberg opened this issue 5 years ago • 5 comments

Right now it is inconvenient to implement continuous action spaces. We need some convenient space types, like Box or something. Unclear if this already exists somewhere or it should go into POMDPModelTools or some new package named with some combination of the words "Box" "Interval" "Real" "Multidimensional" "Vector"

zsunberg avatar May 16 '20 23:05 zsunberg

This might be useful: https://github.com/invenia/Intervals.jl

MaximeBouton avatar May 18 '20 21:05 MaximeBouton

Are simple sets like Box enough? Or do we need support for arbitrary polytopes? It sounds like sampling may become rather tedious at some point.

lassepe avatar May 19 '20 07:05 lassepe

This might be useful: https://github.com/invenia/Intervals.jl

Yes, something like that that someone has already implemented would be great! There are two problems with that particular package though:

  1. It doesn't have rand implemented
  2. It doesn't work like we would want:
julia> [1.5,2.5] in Interval([1,1], [2,2])
true

Are there any other packages?

Are simple sets like Box enough?

Yes, I think boxes are enough for now! if people want polytopes they can write their own space objects easily enough.

The other thing that we might want is cartesian products, i.e. [0,1] x {true, false}

zsunberg avatar May 20 '20 05:05 zsunberg

Maybe we should just make a small implementation for ourselves since hybrid spaces might be already quite a special needs. Product distributions are already supported by Distribitions.jlso the rand interface should be easy to implement for these simple spaces.

lassepe avatar May 20 '20 06:05 lassepe

There is this as well that we could take inspiration from: https://github.com/FluxML/Gym.jl/tree/master/src/Spaces

we could also use that directly but I don't know if we want a dependency on Gym.jl

MaximeBouton avatar May 21 '20 22:05 MaximeBouton