OnlineGo
OnlineGo copied to clipboard
acristescu
Katago stronger than intended
Thanks, this is such a great app and UI. But I think the offline katago is much stronger than intended. I played it against the 7d golaxy bot and they were on completely different levels of thinking. By move 56, golaxy had a .02% (less than 1%) chance of winning and resigned at move 101. Would it be possible to cut the visits in half in the next release? I know with max 4 playouts, it is lightyears ahead of the 4d golaxy bot. sgf of katago 15b4po - golaxy 4d bot match: http://eidogo.com/#xb9fa2Aq golaxy bots: https://www.19x19.com/engine/index sgf of katago 15b4po - golaxy 5d bot - http://eidogo.com/#2SJaN1Tt (misclick so middle game appears as variation. katago (b) still wins, but much closer) or I know it is probably too much too ask but would it be too much work to allow the user to select the number of visits? This would allow all older and newer phones to enjoy the app. But either way, thanks so much! p.s. I know the ogs katago bots need a lot of visits to get a high rank, but that is because katago is several stones weaker in handicap games, though it is much better than other bots. there is a katago bot (katrain-strong) that uses the 15b net with 1 playout and doesn't play handicap games. It is ranked very high.
Hey @portkata thanks for trying things out. How sure are we that the golaxy bot is calibrated correctly? 7d seems like crazy strong for a phone bot!
In any case, in the future versions I will have to certainly tune it down (or allow the user to tune it down). However, using "visits" as a direct setting in the app is something that I'm trying to avoid. It needs to be cleaner for the average user. Furthermore, as you say, even at 1 visit the katago is very strong, the only difference is that it would perhaps sometimes get caught in a ladder.
What we really need to do is to try and implement something like katrain's scaling formula. The app already knows your OGS rank so it could scale katago to the appropriate level.
Out of curiosity, what phone did you try it on and how quickly did it move?
PS: right now it's 20 visits for generating a move, 30 for the hint and territory and 10 for the end-of-game score calculation.
Thanks for the reply! I tried it with a snapdragon 636 processor. onyx boox max e ink ereader. I would guess every move was under 25 seconds, some were much faster. That is such a cool idea incorporating the ogs ranking, wow. it will probably be hard to incorporate the formula though. I think it took akigo several days to get it in his app. Sanderland is also thinking of developing a new bot that will be able to use multiple visits. The golaxy bots are heavily tested on fox, many more games than katrain on ogs. So I think they probably correspond fairly well to fox rankings, which overestimate some, in case in the future you want to test the strength of your bots. Good luck with the project, it is already a great finished product!
I would guess every move was under 25 seconds
Sweet Mary mother of Jesus that's definitely not what I intended. On my 3-years old flagship (S9+) it's about a second. That needs changing, although to be fair that e-reader is definitely not the thing most people would be using.
On the other hand, the fact that a glorified kindle is capable of almost pro-level play is even more impressive and a testament to the work of the katago authors.
Offtopic: I'm actually in the market for the little brother of that e-ink tablet (Boox Nova 3) but it's not available in the UK yet. How happy are you with it?
the onyx devices are awesome. i think the nova 3 is probably the best device for go you can have. i have a likebook mars with a much older processor but same screen as the nova 3 and it is great. the 7.8 inch screen gives you 300 ppi which is great for smartgo books too. i never have to use a computer with the onyx. it does all the computing tasks I could ever need. only downside with my max is ddr3 ram, but nova 3 has ddr4. Really great device.
On the max with katago i can get ddk to 4d bots to play a move within a couple seconds without using kyu_rank. most are within a second. Using kyu_rank, everything is within a second. i use 6b net 1 po - 8 po, then 10b 1po - 5po, then 15b 1po - 4po, all 1 thread. the 20b takes a maybe under 3 seconds for 1 po policy move. It can comfortably beat the golaxy 6d bot. 20b with 2 playout seems stronger than golaxy 7d bot. I did one game of katago 40b 1po policy vs golaxy 8d, I think golaxy was stronger, but katago won, http://eidogo.com/#I4UY0h6J:0,0 Thanks!
you might be interested in this mr. Alex, it blew me away. i did one game of the 15b 1 playout policy against the crazystone pro highest “5d“ setting (maybe shodan in real life?), the 15b policy won by 90! http://eidogo.com/#vVunLVxD . And you were right, the golaxy bots were not calibrated correctly.
@acristescu I played with alot of the nets and i think a weaker unreleased 6b s114 net is probably the best for the app https://d3dndmfyhecmj0.cloudfront.net/g170/neuralnets/selfplayhistory/b6c96-s114663168-d17072879.zip The newest 6b net has weird non human like play, which users won't like. the s114 plays very human like, likes to open on star points, and plays with logic humans can understand. At 100 visits (4 threads so phone usage low on older phones) it's mid dan and will beat 99% of players. it will also play faster than the current setup. Optionally, you can set different strength levels by decreasing by 10 visits at a time, like a slider with strength levels 1-10, but I know that might not be ideal for you. The 1 playout policy is somwhere around 2k (maybe stronger, I don't know). This is the only net of any size i found where you can't tell if you're playing a computer or human player.
Then in the distant future, I was thinking someone else could write the code for this apk have 30 difficulty levels. gnugo levels 1-10, then 10 kyu rank katrain bots (the 6b net is good for this too, hypothetically 6b kyurank 5 to -5) then 10 6b net bots at different visits. i think people would get frustrated just playing the katrain bots, they are really great but not very human like.
