KataGo
KataGo copied to clipboard
lightvector
cannot connect to server via proxy sever
I use a proxy server to visit interset. and I set the https_proxy env by
set https_proxy=http://x.x.x.x:8080/
D:\katago>katago contribute -config contribute.cfg
2021-01-29 11:01:21+0800: Distributed Self Play Engine starting...
2021-01-29 11:01:21+0800: Attempting to connect to server
2021-01-29 11:01:21+0800: isSSL: true
2021-01-29 11:01:21+0800: host: katagotraining.org
2021-01-29 11:01:21+0800: port: 443
2021-01-29 11:01:21+0800: baseResourcePath: /
Uncaught exception: Could not connect to server at https://katagotraining.org/, invalid host or port, or SSL error, or some other httplib error, or no response
but I can you wget to connect https://katagotraining.org/
D:\katago>\wget https://katagotraining.org/
--2021-01-29 11:02:17-- https://katagotraining.org/
Proxy request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'index.html'
index.html [ <=> ] 41.99K 69.2KB/s in 0.6s
2021-01-29 11:02:20 (69.2 KB/s) - 'index.html' saved [42999]
Can you try adding proxyHost=<hostname or ip address of your proxy> and proxyPort=<what port to use for your proxy> into the contribute.cfg?
thanks. those two items are undocumented yet
https://github.com/lightvector/KataGo/blob/master/cpp/configs/contribute_example.cfg
Thanks, added to example config now, which will be included as well in the next release.
could katago also support the https_proxy environment variable like leelazero did
Sure, I could add that. I will consider it lower priority to implement right away since the config option is a workaround, but it can probably be done for next release too.
Can you try adding
proxyHost=<hostname or ip address of your proxy>andproxyPort=<what port to use for your proxy>into thecontribute.cfg?
it works. nice job
is this normal? many minutes passed, stay at this line.
Done tuning
------------------------------------------------------
2021-02-01 08:03:38+0800: Done tuning, saved results to D:\katago/KataGoData/opencltuning/tune8_gpuQuadroK5200_x19_y19_c256_mv10.txt
2021-02-01 08:03:39+0800: OpenCL backend thread 0: Model version 10
2021-02-01 08:03:39+0800: OpenCL backend thread 0: Model name: kata1-b40c256-s5942610176-d1431052518
2021-02-01 08:03:42+0800: OpenCL backend thread 0: FP16Storage false FP16Compute false FP16TensorCores false
2021-02-01 08:03:42+0800: Loaded latest neural net kata1-b40c256-s5942610176-d1431052518 from: katago_contribute/kata1/models/kata1-b40c256-s594261017
6-d1431052518.bin.gz
2021-02-01 08:03:42+0800: Loaded new neural net kata1-b40c256-s5942610176-d1431052518
2021-02-01 08:03:42+0800: Starting game 0 (training) (kata1-b40c256-s5942610176-d1431052518)
2021-02-01 08:03:42+0800: Starting game 1 (training) (kata1-b40c256-s5942610176-d1431052518)
2021-02-01 08:03:42+0800: Starting game 2 (training) (kata1-b40c256-s5942610176-d1431052518)
2021-02-01 08:03:42+0800: Starting game 3 (training) (kata1-b40c256-s5942610176-d1431052518)
does it work ok?
2021-02-01 08:24:04+0800: Maybe predownloading model...
2021-02-01 08:34:11+0800: Finished game 2 (training), uploaded sgf katago_contribute/kata1/sgfs/kata1-b40c256-s5942610176-d1431052518/67483C2F8E25CAE5
.sgf and training data katago_contribute/kata1/tdata/kata1-b40c256-s5942610176-d1431052518/201211D988B82A47.npz (15 rows)
2021-02-01 08:34:11+0800: Starting game 4 (training) (kata1-b40c256-s5942610176-d1431052518)
2021-02-01 08:48:23+0800: Maybe predownloading model...
That looks pretty normal, assuming you specified that it would play 4 games at a time in parallel. Please check: https://katagotraining.org/contributions/ and find your username, and you can see if the games were uploaded successfully and if they look like they good games with reasonable moves.
my uploaded game is a 9x9 game, is it ok? https://katagotraining.org/sgfplayer/training-games/3747163/
https_proxy environment variable should now be supported in the tip of stable branch, which will go out next release as well.
https://github.com/lightvector/KataGo/commit/1756c1d04188f05fe7b2ebd6de8a832a283209cb
It says everything is loaded fine and tuned. It then starts the training (set to 40 games) it seems to start those and then goes to Maybe predownloading model . . . endlessly?
Something is wrong. Please stop running it for now and see if you can just run normally using a GUI program like Lizzie or KaTrain or q5go to review or analyze games? (seeing if you can run it to do anything else and get reasonable moves or analysis will give some idea as to whether there are other major issues).
This is the relevant bit of code. https://github.com/lightvector/KataGo/blob/master/cpp/command/contribute.cpp#L845
Which I don't see how it's even possible to get an output like yours where it outputs multiple messages per second, unless somehow on your system, std::this_thread::sleep_for doesn't actually sleep. And if that's the case, it makes me wonder what else might be not working with KataGo on your system. What operating system are you using, and are you using the release executable or something custom-compiled?
Katago works fine in Lizzie (as I have used it for a lot of game analysis and matches against Leela. It also works great in Sabaki for self playing against itself. I have Windows 10, my GPU is RTX 2080 Ti. I thought I would try to share in the process of contributing and this is what occurred. I am using the released version Open CL.
I was going to ask how strong do we think Katago is relative to AlphaGo? What will it take to get Katago even stronger to where it would make this current version look weak? Thanks
Thanks for the log and other info, I'll see if I can figure out what's going on in the next few days and I might post here a modified version of KataGo with some more debug info for you to try to run at some point.
Nobody knows for sure, but I think there is a good chance that KataGo is stronger than AlphaGo used to be, as long as you equalize the hardware used at match time. KataGo has architecture and training improvements that were not known at AlphaGo's time, and what should be slightly better search methods. In the last year, KataGo has improved about 200 Elo in networks alone, plus on the order of another 100 Elo in search improvements.
Beyond just training longer (which continues to help), there are still some known algorithmic directions to improve further as well, if we can get the time and resources to implement them.
I can not find an answer to this issue, but it may be me. When Katago is running if I have my speakers on it make this crazy screeching sound that sounds like it is on an infinite loop screeching white noise, very annoying.
Still have no idea how your result is possible except that perhaps on your machine, sleeping a thread somehow doesn't make it sleep. If you're interested to investigate, here is an executable that you can use to test this, try running:
katago.exe runsleeptest
Does it actually sleep for the desired amounts of time?
katagosleeptest.zip
As for screeching on your speakers, searching online suggests this is an issue of an unstable power supply when the GPU is running, or something like that. See: https://www.nvidia.com/en-us/geforce/forums/3d-vision/41/249476/nvidia-ground-loop-noise-under-load/ https://linustechtips.com/topic/1165042-gpu-noise-playing-through-speakers/ https://www.nvidia.com/en-us/geforce/forums/game-ready-drivers/13/187956/cracklingcracking-sound-when-playing-games-through/ https://www.reddit.com/r/buildapc/comments/7k59pn/graphics_card_noise_in_speakers/
Are you using the executable I linked in the above post? Not the normal katago.exe.
Hmmm, thanks. That's really strange, looks like the sleep test works fine. So I have no idea what could possibly be happening with your katago when attempting to contribute. I don't know of anyone else who reported something similar, so I don't know what could be causing it.
If you want to use this exe going forward for your own analysis purposes, feel free. It won't work for contribute on katagotraining.org, but in all other respects it's almost the same as v1.9.1.
As for testing out contribute, maybe I can compile a special version with more debug code around the predownload loop to see more what's going on. There is a test server "test.katagodistributed.org" that is designed to allow running any different test or debugging versions without affecting the official run. If you think it's worthwhile, find me on the LZ discord and PM me (https://discord.gg/bqkZAz3) or otherwise let me know a way to contact you privately, and I'll give you a username and password on the test site, and an exe with more debugging info that you can try to run for longer and see if you reproduce it being stuck reporting predownload loop every second.
Also, I hope you find a solution for your GPU speaker noise issue.