KataGo
KataGo copied to clipboard
lightvector
Why does this no-variation opening appear so frequently in KataGo’s training games?
There is a no-variation opening in training games, and it appears very very frequently.
https://katagotraining.org/sgfplayer/training-games/61976462/
https://katagotraining.org/sgfplayer/training-games/60902084/
Why does this opening appear so frequently in KataGo’s self-play game records, with almost no variation from the 1st to the 44th move every time it occurs? It has been like this for quite a long time, maybe even earlier before 60B. Could this affect KataGo’s training?
The similar issue has been around since the era of Leela 40B.
Is there any way to technically reduce the occurrence frequency of the opening? For example, by increasing the self-play temperature or automatically filtering out these repetitive positions during training? And I suggest continuing to increase the diversity of openings in self-play games, exploring more possibilities, and avoiding overfitting.
https://katagotraining.org/sgfplayer/training-games/63975130/
This is another variant of the opening, which now appears with a very high frequency. It is almost no-change from the 1st to the 40th move.
Maybe it is a pretty opening? (although I don't think so) What are the most frequent openings that LeelaZero 40 series weights trained, the same as KataGo? Does this sort of behaviors stop LeelaZero from getting more powerful? From my rough view, openings could easily be changed by set kata-param wideRootNoise.
Maybe it is a pretty opening? (although I don't think so) What are the most frequent openings that LeelaZero 40 series weights trained, the same as KataGo? Does this sort of behaviors stop LeelaZero from getting more powerful? From my rough view, openings could easily be changed by set kata-param
wideRootNoise.
Leela’s repetitive opening moves are much more than KataGo’s.
Perhaps the noise in training games is too low? I’m worried that too many repetitive moves in the opening phase may affect the improvement of KataGo's ELO.
@poptangtwe
KataGo's search is significantly less diverse with too little fluctuation.
Opening of the 7X7 board JP rules
B(A6) on move 13 is extremely promising and appears in the opening book, but the policy weighting is less than 1% so it is completely ignored in the default setting .
https://katagotraining.org/sgfplayer/training-games/64162195/
This is another variant of this opening, which appears with a high frequency recently. It is almost no-change from the 1st to the 27th move.
https://katagotraining.org/sgfplayer/training-games/64192809/
For this diagonal shape, it seems that KataGo has fallen into overfitting or a local optimum. In this game, the 3-3 point joseki in the top right corner is different from usual, but KataGo still played this highly similar double hane joseki. It is almost no-change from the 17th to the 38th move.
I hope to increase the diversity of moves for this specific diagonal shape.
Thanks for the discussion. I'll investigate a little and consider what can be done about this. I don't think it's harmful to the training, but having a lot of repetition in the openings does probably indicate that we could be a bit more diverse and efficient with compute with adjusted parameters for exploration or more forced starting positions with different stones.
https://katagotraining.org/sgfplayer/rating-games/1356275/
it occurs again and again.
https://katagotraining.org/sgfplayer/training-games/64620011/
This game, witch is better than the previous games, is almost no-change from the 1st to the 23th move.
https://katagotraining.org/sgfplayer/training-games/64623420/
This openning is so boring.
If you can solve this problem, I would be very grateful.
I reviewed probably around 50 different 19x19 training games and I haven't yet seen this opening pattern even one time. You've found some instances of this pattern in training games, which you linked above, so I'm sure that it's probably still one of the most frequent openings, but how frequent it is actually in absolute terms? Is it as high as a few percent, or is it less?
Most training games I see are very different and seem pretty diverse in their openings, for example: https://katagotraining.org/sgfplayer/training-games/64745059/ https://katagotraining.org/sgfplayer/training-games/64744947/ https://katagotraining.org/sgfplayer/training-games/64744859/ https://katagotraining.org/sgfplayer/training-games/64744821/
I found some games share similar joseki, but are still very different globally, such as this one,: https://katagotraining.org/sgfplayer/training-games/64744804/
And also of course there are other games mixed in besides regular 19x19 games: SGF positions, different board sizes, handicap games, etc, also adding variety. So my inclination right now seeing the an actual sample of games is to make no changes. It doesn't seem to me like there is any problem to fix.
I haven't checked rating games, it's possible that the rating games have this opening much more frequently. The rating games have far less randomization because too much randomization would result in a larger proportion of unfair games. However, it's difficult to justify making any changes to rating games because if the rating game settings are adjusted, then the ratings from the new games become incomparable to those of the old games - it's not proper to mix games into the same rating system when the games were played with different parameters and conditions.
I reviewed probably around 50 different 19x19 training games and I haven't yet seen this opening pattern even one time. You've found some instances of this pattern in training games, which you linked above, so I'm sure that it's probably still one of the most frequent openings, but how frequent it is actually in absolute terms? Is it as high as a few percent, or is it less?
Most training games I see are very different and seem pretty diverse in their openings, for example: https://katagotraining.org/sgfplayer/training-games/64745059/ https://katagotraining.org/sgfplayer/training-games/64744947/ https://katagotraining.org/sgfplayer/training-games/64744859/ https://katagotraining.org/sgfplayer/training-games/64744821/
I found some games share similar joseki, but are still very different globally, such as this one,: https://katagotraining.org/sgfplayer/training-games/64744804/
And also of course there are other games mixed in besides regular 19x19 games: SGF positions, different board sizes, handicap games, etc, also adding variety. So my inclination right now seeing the an actual sample of games is to make no changes. It doesn't seem to me like there is any problem to fix.
I haven't checked rating games, it's possible that the rating games have this opening much more frequently. The rating games have far less randomization because too much randomization would result in a larger proportion of unfair games. However, it's difficult to justify making any changes to rating games because if the rating game settings are adjusted, then the ratings from the new games become incomparable to those of the old games - it's not proper to mix games into the same rating system when the games were played with different parameters and conditions.
Based on the training games I have seen, I estimate that roughly 1 in 200 to 1 in 400 of the 19x19 normal-type games start with this opening.
https://katagotraining.org/sgfplayer/training-games/65037566/
It occurs again within s788.
Thanks for the estimate. If it's perhaps only 0.2% of the total games (including all game types), what is the concern? Are we worried about some kind of overfitting? It seems like by move 50 there are already several variants of this opening and the games diverge further afterward, this is reflected in the white winrate of the game (around 63% given the komi and rules) showing there's still quite a bit of uncertainty in the outcome. Do we think this game is a sign that there is not much diversity in the data? It seems to me so far that there's still plenty of diversity in other games.