fishtest
fishtest copied to clipboard
glinscott
non-ascii paths.
an exchange in discord about a crashing worker getraideBFF https://tests.stockfishchess.org/tests/view/61d800f1f5fd40f357469b0a revealed that that worker was running into https://github.com/official-stockfish/Stockfish/issues/3424
This can't be easily fixed in SF, but maybe the worker code detect the fact it is running in a path that contains non-ascii characters (cyrillic) and detect it?
I've documented this limitation here https://github.com/glinscott/fishtest/wiki/Running-the-worker:-overview#install-the-worker
I guess it is a windows problem. Windows cannot handle UTF-8 path names (the most common encoding on the planet). So one has to convert first to UTF-16.
First result on Google https://stackoverflow.com/questions/7153935/how-to-convert-utf-8-stdstring-to-utf-16-stdwstring
yes, and the SF code needs to use the right types for filenames in all places etc... it is definitely not completely trivial to get right.
The worker could send the short path name to SF: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getshortpathnamew if non-asci characters are detected in a path name (is it only the NNUE path?).
I think it is not worth trying to do it from the worker. Things in SF it self might break (like it searches for the net in a couple of directories).
Ok. Not allowing non asci characters in filenames probably excludes a sizable part of the world's population though. Nowadays people expect unicode to "just work".
EDIT. Fixed typo.
yes, I agree, so well tested patches for SF would be welcome there :-) For whatever reason the Lc0 people said it is really non-trivial, so I take their word for it.
I really think this is fixable on the worker's side. This is the cuplrit
option.EvalFile=/home/fishtest/fishtest/worker/testing/nn-13406b1dcbe0.nnue
The current directory is /home/fishtest/fishtest/worker/testing. So there doesn't seem to be any need to supply a path at all.
vector<string> dirs = { "<internal>" , "" , CommandLine::binaryDirectory , stringify(DEFAULT_NNUE_DIRECTORY) };
If I interpret this correctly it will first search in the current directory. Which is what we want.
it will first use the embedded net, after that it will search the directory of the binary.... that would probably go wrong.
No, the empty string "" is in the second place. That should be the current directory.
If it doesn't work we can also embed the net (first copy the downloaded/cached net to the build directory).
we specifically didn't do that since that would make the cache of the binaries large in size.
It is fixed on the worker’s side now. However it might still be a good idea to trace the cause of the crash in SF and print an error message instead. Some system call used by SF must return with an error.
I guess SF doesn't actually crash but just doesn't find the net and then exits. That would be perfectly fine of course.