depthai-core
depthai-core copied to clipboard
env vars with non-ascii paths fail on Windows
depthai paths are also stored in environment variables. Therefore apis that read env vars need to use Windows unicode apis wchar
to get the full UTF-16 which can then be converted into dai::path or UTF-8.
Found while fixing #352 PR #384 enables Unicode paths in API signatures, if/ofstream, and spdlog. A second PR should enable Unicode from environment variables...which naturally needs the first PR's fixes.
Setup
- all on Windows
Discussion
The os manages the storage of the different char sets (ascii, utf-8, utf-16, etc.). When an app needs the env var value, it calls an os API. On Windows...it needs to call the UTF-16 wide character APIs for environment variables to get Unicode text.
A Windows customer in a country/language who's glyphs do not fit in ASCII, e.g. Japan, China, UAE, Germany, etc. stores a filename or path, using their native language, into env vars that depthai reads. depthai calls the API to get the env var.
// runtime fail: underlying utility calls ascii OS api, Windows provides best-fit value, the path is corrupt/unusable
auto fwBinaryPath = utility::getEnv("DEPTHAI_DEVICE_BINARY");
// compile fail: underlying utility calls the ascii OS api which returns a std::string, it should return a Unicode std::wstring
std::wstring fwBinaryPath = utility::getEnv("DEPTHAI_DEVICE_BINARY");
I believe the utility::getEnv() call chain should always return wstring on Windows. Making the caller responsible for managing any unicode/ascii/number convert. After the getEnv() is fixed, then this is possible...
// works: the getEnv() code has been updated to return wstring
auto fwBinaryPath = utility::getEnv("DEPTHAI_DEVICE_BINARY");
// works: the getEnv() code has been updated to return wstring and dai::Path can be directly constructed with it
dai::Path fwBinaryPath = utility::getEnv("DEPTHAI_DEVICE_BINARY");
There ?may? be some cascading work to ensure things that parse the env var wstring are not assuming char
. Templates and/or the typical MSVC overloads that can accept wchar in functions can likely address these.
Quick look at spdlog's code, they are partially aware of this issue. I see macro defines like SPDLOG_WCHAR_TO_UTF8_SUPPORT
, SPDLOG_WCHAR_FILENAMES
but I don't see they dealt with Windows for the env vars.
Originally posted by @diablodale in https://github.com/luxonis/depthai-core/issues/352#issuecomment-1049062049