Problems with non-asciil characters in paths

[vondele]

an EvalFile cannot be used as a setoption if the path contains certain non-ascii characters.

for example setoption name EvalFile value C:\Users\Owner\Desktop\русский\nn-62ef826d1a6d.nnue does not work

similar issue in Leela, we might look there for solutions.. https://github.com/LeelaChessZero/lc0/issues/1474

[snicolet]

This stackoverflow post seems to have some related info: https://stackoverflow.com/questions/30829364/open-utf8-encoded-filename-in-c-windows

Not sure if we should care much about it in Stockfish

[AndrewTCEC]

As an actual finder of this problem, I say that I will wait as long as it takes to get it fixed and if you are not caring so much it's okay, it is a minor problem in the face of making the engine stronger. So take your time. I will adjust for ascii-only paths meanwhile. And another softening fact is that it takes place in SF-console only, using Fish in Arena does not cause any troubles (for some reason).

[dsmsgms]

std::string is just bytes, Arena must be sending the correct encoding to standard input and console is using UTF-8.

[gvreuls]

It seems that UTF-8 console input is buggy at least for the dotnet runtime, if this is related I'm not sure we should code around it.

[dsmsgms]

I cannot reproduce on this my system (Linux). Someone else has to check if we are going to find a potential fix. See link for an attempt. https://github.com/dsmsgms/Stockfish/tree/non_utf8_path/