mcts is reset after every episode?

[tueboesen]

I am not an expert in mcts at all, but I have been playing around with your code and I noticed that in the learn method the mcts is reset after every single episode. Meaning the mcts is reset after every single game when it is generating data to train on. Hence the mcts never grows beyond the information of a single game, this seems weird to me since I would expect you to want to keep the mcts and continue to evolve it and only reset it once the neural network has been trained on the data generated?

Is this really intended?

[arnemileswinter]

If i understand your question correctly, it's what they do in AlphaGoZero too:

The search tree is reused at subsequent time-steps: the child node corresponding to the played action becomes the new root node; the subtree below this child is retained along with all its statistics, while the remainder of the tree is discarded.

so i think it is intended. Have you tried reusing the tree? (I would guess that the memory footprint will be ginormous, though)

[cyshe]

args.numMCTSSims games are played when calling getActionProb for each MCTS.