boardgame.io
boardgame.io copied to clipboard
MCTS Bot, stopping when an objective is met
Here's the playout code from the MCTS bot at ai/mcts-bot.ts:
for (let i = 0; i < playoutDepth && state.ctx.gameover === undefined; i++) {
const { G, ctx } = state;
let playerID = ctx.currentPlayer;
if (ctx.activePlayers) {
playerID = Object.keys(ctx.activePlayers)[0];
}
const moves = this.enumerate(G, ctx, playerID);
// Check if any objectives are met.
const objectives = this.objectives(G, ctx, playerID);
const score = Object.keys(objectives).reduce((score, key) => {
const objective = objectives[key];
if (objective.checker(G, ctx)) {
return score + objective.weight;
}
return score;
}, 0);
// If so, stop and return the score.
if (score > 0) {
return { score };
}
if (!moves || moves.length == 0) {
return undefined;
}
const id = this.random(moves.length);
const childState = this.reducer(state, moves[id]);
state = childState;
}
My understanding of this code is, the bot uses each objective's checker function to see if the objective is met. If even 1 objective is met, the bot immediately stops searching that branch and does not go further to finish playout depth. Is this correct?
I suppose this helps in games where objectives can be continually met and then removed. For example, in Connect 4, when it's desirable to get 3 in a row with an empty space adjacent to that row, that can be an objective, and of course the other player will immediately respond so that your objective is no longer met.
But what about more complex games? I feel that if you're playing, say, chess, an "objective" might be to remove the opponent's queen from the board. Pretty simple objective - check G, if opponent's queen is gone, that's an objective met. But once you take their queen, that objective is met forever. So from now on the bot will pretty much only look 1 move ahead from now on, right?
Should there be an option on the MCTS bot, or perhaps just another bot, that, instead of looking for at least 1 objective and then stopping, goes down the tree, and "scores" the final state of G? (It could even be a negative score, if you lost your queen, etc.)
Chess and more complex games achieve this by having a score function to account for more desirable boards vs less desirable boards. The objective is always to win the game