if the trajectory stays n the terminal state (for a limited number of times)

[ArezooAalipanah]

hi thank you sooooooo much for this amazing repo. I have been trying to build mu own environment but I faced some issues. what if we have something like this : going from s0 to s1 to s2 and then staying in s3 for ever (I changed the value iteration so now my trajectories are all 50 steps ) so my svf is something like(1,1,1,47, 0,...,0) However I am facing some difficulties. my zs and za start getting so big and then they become nan. and this ends in my omega to be nan as well I was wondering if you have any idea how I can fix it? and what is the problem. I am reading Dr.Zeibart's thesis but still have no clue how to tackle such problem(since z_terminal is 1 I am thinking maybe that results in the problem) if you have any idea I would be so grateful if you share your thoughts Thanks again

[ArezooAalipanah]

here is a bit more info : my trajectory (I made it of len 40 this time) the first iteration with initialization of 1 next array is my parameters after first iteration however after second iteration they all end up being nan

[qzed]

Hi, I'm sorry for the long silence. I will likely need some time to look into this, and as I was a bit busy with work-related things lately I never got around to it. Things should be less stressful now, so I will try to look into it this weekend.

[ArezooAalipanah]

thank you so much. I made some modifications, like normalizing the rewards and weights each time to avoid going to infinity, but I still have to keep number of iterations limited since it will never converge. I am looking forward for your insight as well _