POMDPs.jl icon indicating copy to clipboard operation
POMDPs.jl copied to clipboard

TMaze + DiscreteUpdater errors

Open FlyingWorkshop opened this issue 10 months ago • 1 comments

Steps to recreate:

using POMDPs, POMDPTools, POMDPModels

pomdp = TMaze(1, 0.99)
updater = DiscreteUpdater(pomdp)
b0 = DiscreteBelief(pomdp, [1.0, 0.0, 0.0, 0.0, 0.0])
a = 1
o = 2
b = update(updater, b0, a, o)

Error:

ERROR: LoadError: Failed discrete belief update: new probabilities sum to zero.

b = DiscreteBelief{TMaze, Union{TerminalState, TMazeState}}(TMaze(1, 0.99), Union{TerminalState, TMazeState}[TMazeState(1, :north), TMazeState(1, :south), TMazeState(2, :north), TMazeState(2, :south), TerminalState()], [1.0, 0.0, 0.0, 0.0, 0.0])
a = 1
o = 2

Failed discrete belief update: new probabilities sum to zero.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] update(bu::DiscreteUpdater{TMaze}, b::DiscreteBelief{TMaze, Union{TerminalState, TMazeState}}, a::Int64, o::Int64)
   @ POMDPTools.BeliefUpdaters ~/.julia/packages/POMDPTools/7Rekv/src/BeliefUpdaters/discrete.jl:133
 [3] top-level scope
   @ ~/VSCodeProjects/BeliefCompression/arena/tmaze_benchmark.jl:8
in expression starting at /Users/logan/VSCodeProjects/BeliefCompression/arena/tmaze_benchmark.jl:8

Is this expected? You can also trigger the error with this:

using POMDPs, POMDPTools, POMDPModels
using DiscreteValueIteration

pomdp = TMaze(1, 0.99)
updater = DiscreteUpdater(pomdp)
b = initialize_belief(updater, initialstate(pomdp))
for _ in 1:10
    for a in actions(pomdp, b)
        for o in observations(pomdp)
            b_new = update(updater, b, a, o)
            global b = b_new
        end
    end
end

FlyingWorkshop avatar Apr 01 '24 23:04 FlyingWorkshop

I think this is because you can't observe 2 when you are in state 1. Since b0 assigns probability 1 to being in state 1, then this will fail as expected.

mykelk avatar Apr 06 '24 15:04 mykelk