minigo icon indicating copy to clipboard operation
minigo copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

SWA attempt

Open sethtroisi opened this issue 6 years ago • 2 comments

I'm trying out SWA

Write-up: https://github.com/glinscott/leela-chess/issues/159

Original post: https://github.com/gcp/leela-zero/issues/814

Code: https://github.com/sethtroisi/minigo/tree/swa?files=1

https://github.com/sethtroisi/minigo/blob/swa/swa.py

I have ringmaster test running right now between swa-2, swa-4, swa-8, swa-10, and 000303-olympus

sethtroisi avatar Jun 20 '18 17:06 sethtroisi 

minigo-cc-swa-2 v minigo-cc-swa-4 (100/100 games)
board size: 19   komi: 7.5
                  wins              black         white       avg cpu
minigo-cc-swa-2     39 39.00%       16 32.00%     23 46.00%    111.23
minigo-cc-swa-4     61 61.00%       27 54.00%     34 68.00%    111.48
                                    43 43.00%     57 57.00%

minigo-cc-swa-4 v minigo-cc-swa-8 (100/100 games)
board size: 19   komi: 7.5
                  wins              black         white       avg cpu
minigo-cc-swa-4     52 52.00%       25 50.00%     27 54.00%    114.76
minigo-cc-swa-8     48 48.00%       23 46.00%     25 50.00%    114.81
                                    48 48.00%     52 52.00%

minigo-cc-swa-8 v minigo-cc-swa-10 (100/100 games)
board size: 19   komi: 7.5
                   wins              black         white       avg cpu
minigo-cc-swa-8      52 52.00%       26 52.00%     26 52.00%    116.53
minigo-cc-swa-10     48 48.00%       24 48.00%     24 48.00%    116.51
                                     50 50.00%     50 50.00%

minigo-cc-swa-2 v minigo-cc-303 (100/100 games)
board size: 19   komi: 7.5
                  wins              black         white       avg cpu
minigo-cc-swa-2     47 47.00%       29 58.00%     18 36.00%    112.41
minigo-cc-303       53 53.00%       32 64.00%     21 42.00%    112.34
                                    61 61.00%     39 39.00%

minigo-cc-swa-10 v minigo-cc-303 (100/100 games)
board size: 19   komi: 7.5
                   wins              black         white       avg cpu
minigo-cc-swa-10     61 61.00%       25 50.00%     36 72.00%    111.15
minigo-cc-303        39 39.00%       14 28.00%     25 50.00%    110.77
                                     39 39.00%     61 61.00%

sethtroisi avatar Jun 21 '18 03:06 sethtroisi

Per LZs implementation it seems like we should run another couple of batches of training so that batch norms can be recalculated

https://github.com/gcp/leela-zero/commit/94f48bb10b20d2267fcd3b9b0531f00c51907909#diff-533b2cfc6eb41b6d5b8954e445851082R453

sethtroisi avatar Jun 24 '18 17:06 sethtroisi