Possible ways to replicate a env.

[smgreat]

I am using the MCTS algorithm and need to replicate a env object at current step . And then run the copies of the env object separately. I have tried copy.deepcopy() and pickle.dumps(), but it got the following error, TypeError: can't pickle _thread.lock objects Does anyone have suggestions or solutions?

[andrewcoh]

Hi @smgreat

If I understand correctly, you are trying to replicate envs on the python side i.e. in the LLAPI to perform MCTS like rollouts?

Simply copying a Unity build like this won't work i.e. it wont instantiate an exact copy of the current env at the moment of copy. It's hard to make a recommendation without knowing more because "rolling back" a unity simulation to a previous state is not possible in general due to physical non-determinism.

Are you trying to do strict MCTS or do you want to do a MuZero like approach?

[smgreat]

Thank you so much for the reply @andrewcoh .

You understand correctly. I'm trying to use a simple MCTS on a match3 game, when I try to restore the env state, deep copy won't work. So I was wondering if there is a way to directly modify the env via the API? Or for match3 games, is there any other algorithm recommendation when the env cannot be restore?

Many Thanks!