Enable HDK on Windows

[ienkovich]

Enabling includes successful execution of all OmniSci and HDK tests and integration with Modin.

[alexbaden]

Blocked by problems w/ the cython build:

running build_ext
building 'pyhdk._common' extension
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DBOOST_LOG_DYN_LINK=1 -DWIN_32=1 -IC:\Users\alexb\miniconda3\envs\omnisci-dev\lib\site-packages\numpy\core\include -IC:\Users\alexb\miniconda3\envs\omnisci-dev\lib\site-packages\pyarrow\include -IC:/Users/alexb/Projects/hdk/python -IC:/Users/alexb/Projects/hdk -IC:/Users/alexb/Projects/hdk/omniscidb -IC:/Users/alexb/Projects/hdk/omniscidb/ThirdParty -IC:/Users/alexb/Projects/hdk/omniscidb/ThirdParty/rapidjson -IC:/Users/alexb/miniconda3/envs/omnisci-dev/Library/include -IC:/Users/alexb/Projects/hdk/python -IC:\Users\alexb\miniconda3\envs\omnisci-dev\include -IC:\Users\alexb\miniconda3\envs\omnisci-dev\include /EHsc /TpC:/Users/alexb/Projects/hdk/python/pyhdk/_common.cpp /Fobuild\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk/_common.obj /std:c++17
_common.cpp
C:\Users\alexb\Projects\hdk\omniscidb\QueryEngine\Descriptors/CountDistinctDescriptor.h(56): warning C4334: '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?)
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\lib\site-packages\pyarrow /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\Library\lib /LIBPATH:C:/Users/alexb/Projects/hdk/build/python /LIBPATH:. /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/Calcite/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/ArrowStorage/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/QueryEngine/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/DataMgr/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/SchemaMgr/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/Shared/Release /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\libs /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\libs /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\PCbuild\amd64 arrow.lib arrow_python.lib QueryEngine.lib Shared.lib /EXPORT:PyInit__common build\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk/_common.obj /OUT:build\lib.win32-3.8\pyhdk\_common.cp38-win_amd64.pyd /IMPLIB:build\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk\_common.cp38-win_amd64.lib
   Creating library build\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk\_common.cp38-win_amd64.lib and object build\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk\_common.cp38-win_amd64.exp
Shared.lib(DateTimeParser.obj) : error LNK2001: unresolved external symbol __CxxFrameHandler4
QueryEngine.lib(WindowFunctionIR.obj) : error LNK2001: unresolved external symbol __CxxFrameHandler4
QueryEngine.lib(RexSubQueryIdCollector.obj) : error LNK2001: unresolved external symbol __CxxFrameHandler4

repeats

I am using the same x86_x64 cross compiler for both cmake and cython, so am not sure where this mismatch is coming from. We do not appear to have any control over the Cython internals.

I am currently trying this on Windows 11, which may be part of the problem. But if our delivery mechanism is going to be conda, perhaps we should be trying mingw instead of msvc?

[alexbaden]

Visual Studio 2022 build tools fails in compilation of the library:

C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.32.31326\include\yvals_core.h(599,2): error G69703BD3: STL1000: Unexpected compiler version, expected Clang 13.0.0 or newer.

Attempting a conda solve with clang-13.

[alexbaden]

Same issue with latest msvc and llvm-13. Going to pursue MinGW.

[aregm]

https://github.com/conda-forge/heavydb-ext-feedstock is functional - need to use this for enabling Windows build.

[alexbaden]

I don't think that's particularly useful given divergence in dependencies and code bases. What were you hoping to learn from it?

[Garra1980]

I don't think that's particularly useful given divergence in dependencies and code bases. What were you hoping to learn from it?

+1 I doubt it will help hdk enabling

[aregm]

Ok. Then we're back to square 1.