Building Python bindings in Windows - missing static library

[kocurvik]

Hello,

I managed to install the python bindings on Windows, but there was an issue during the process. I figured this may save someone some time.

The first two lines for the installation:

cmake -S . -B _build/ -DPYTHON_PACKAGE=ON -DCMAKE_INSTALL_PREFIX=_install
cmake --build _build/ --target install -j 8

run fine without errors. However when I ran the next line:

cmake --build _build/ --target pip-package

I got the following error:

Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  PoseLib.vcxproj -> D:\Research\code\PoseLib\_build\PoseLib\Debug\PoseLibd.lib
  -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044.
  -- BUILD_SHARED_LIBS: OFF
  -- CMAKE_CONFIGURATION_TYPES: Release;Debug;MinSizeRel;RelWithDebInfo
  -- CMAKE_GENERATOR: Multi-config
  -- CMAKE_GENERATOR: Visual Studio 16 2019
  -- CMAKE_REGISTRY_FOLDER: OFF
  -- Python_EXECUTABLE: C:/Anaconda3/envs/torch/python.exe
  -- Configuring done
  -- Generating done
  -- Build files have been written to: D:/Research/code/PoseLib/_build
  running bdist_wheel
  running build
  running build_ext
  building 'poselib' extension
  "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe" /c /
  nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Anaconda3\envs\torch\lib\site-packages\pybind11\include -ID:/Research/code/PoseL
  ib/_install\include -ID:/Research/code/vcpkg/installed/x64-windows/include/eigen3 -IC:\Anaconda3\envs\torch\include -
  IC:\Anaconda3\envs\torch\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29
  .30133\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\inc
  lude" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\
  include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (
  x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwi
  nrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include" "-IC:\Progra
  m Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\atlmfc\include" "-IC:\Program Files (x
  86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\Include
  \10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\VS\UnitTest\include
  " "-IC:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\Includ
  e\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\winrt" "-IC:\Program Files (x86
  )\Windows Kits\10\Include\10.0.19041.0\cppwinrt" -IInclude\um /EHsc /TpD:/Research/code/PoseLib\pybind\pyposelib.cpp
  /Fobuild\temp.win-amd64-cpython-38\Release\Research/code/PoseLib\pybind\pyposelib.obj /std:c++latest /EHsc /bigobj -s
  td=c++17 -march=native -ffast-math -fno-unsafe-math-optimizations
cl : Command line warning D9002: ignoring unknown option '-std=c++17' [D:\Research\code\PoseLib\_build\pybind\pip-packa
ge.vcxproj]
  pyposelib.cpp
cl : Command line warning D9002: ignoring unknown option '-march=native' [D:\Research\code\PoseLib\_build\pybind\pip-pa
ckage.vcxproj]
cl : Command line warning D9002: ignoring unknown option '-ffast-math' [D:\Research\code\PoseLib\_build\pybind\pip-pack
age.vcxproj]
cl : Command line warning D9002: ignoring unknown option '-fno-unsafe-math-optimizations' [D:\Research\code\PoseLib\_bu
ild\pybind\pip-package.vcxproj]
D:/Research/code/PoseLib/_install\include\PoseLib/robust/estimators/absolute_pose.h(44): warning C4267: 'argument': con
version from 'size_t' to 'int', possible loss of data [D:\Research\code\PoseLib\_build\pybind\pip-package.vcxproj]
D:/Research/code/PoseLib/_install\include\PoseLib/robust/estimators/absolute_pose.h(154): warning C4267: 'argument': co
nversion from 'size_t' to 'int', possible loss of data [D:\Research\code\PoseLib\_build\pybind\pip-package.vcxproj]
D:/Research/code/PoseLib/_install\include\PoseLib/robust/estimators/relative_pose.h(44): warning C4267: 'argument': con
version from 'size_t' to 'int', possible loss of data [D:\Research\code\PoseLib\_build\pybind\pip-package.vcxproj]
D:/Research/code/PoseLib/_install\include\PoseLib/robust/estimators/relative_pose.h(112): warning C4267: 'argument': co
nversion from 'size_t' to 'int', possible loss of data [D:\Research\code\PoseLib\_build\pybind\pip-package.vcxproj]
D:/Research/code/PoseLib/_install\include\PoseLib/robust/estimators/homography.h(44): warning C4267: 'argument': conver
sion from 'size_t' to 'int', possible loss of data [D:\Research\code\PoseLib\_build\pybind\pip-package.vcxproj]
  "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe" /n
  ologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Anaconda3\envs\torch\libs /LIBPATH:
  C:\Anaconda3\envs\torch /LIBPATH:C:\Anaconda3\envs\torch\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Vis
  ual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual
   Studio\2019\Community\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.1
  9041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" "/LIBPATH:C:\Program Files
   (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Mic
  rosoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\atlmfc\lib\x64" "/LIBPATH:C:\Program Files (x86)\Micros
  oft Visual Studio\2019\Community\VC\Auxiliary\VS\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.1
  9041.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\VS\UnitTest\lib
  " "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64" /LIBPATH:lib\um\x64 /EXPORT:PyInit_poseli
  b build\temp.win-amd64-cpython-38\Release\Research/code/PoseLib\pybind\pyposelib.obj D:/Research/code/PoseLib/_instal
  l\lib\libPoseLib.a /OUT:build\lib.win-amd64-cpython-38\poselib.cp38-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpytho
  n-38\Release\Research/code/PoseLib\pybind\poselib.cp38-win_amd64.lib
LINK : fatal error LNK1181: cannot open input file 'D:\Research\code\PoseLib\_install\lib\libPoseLib.a' [D:\Research\co
de\PoseLib\_build\pybind\pip-package.vcxproj]
CUSTOMBUILD : error : command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.2
9.30133\\bin\\HostX86\\x64\\link.exe' failed with exit code 1181 [D:\Research\code\PoseLib\_build\pybind\pip-package.vc
xproj]
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241
,5): error MSB8066: Custom build for 'D:\Research\code\PoseLib\_build\CMakeFiles\b1cfb236130b9e0f20d3c778ae1a2caf\pip-p
ackage.rule' exited with code 1. [D:\Research\code\PoseLib\_build\pybind\pip-package.vcxproj]

I managed to resolve the issue by opening the PoseLib.sln in Visual Studio and running the Release build. Then I copied the file _build\PoseLib\Release\PoseLib.lib to _installation\lib\libPoseLib.a and then I was able to successfully run the last two CMake commands:

cmake --build _build/ --target pip-package
cmake --build _build/ --target install-pip-package

I guess the issue might be related to the Windows build creating a .lib file instead of .a file? Also it seems that the default build is Debug instead of Release in the pybind README.

[pablospe]

Could you try running scripts/build_python_package.sh?

[pablospe]

Also check you could build the package inside docker? See this PR https://github.com/vlarsson/PoseLib/pull/52.

[kocurvik]

Could you try running scripts/build_python_package.sh?

I have seen the pull request even before getting merged and it did not work. I have tried it now again. There are still the same two problems. The build gets built as debug resulting in _install_python/lib/LibPose.libd to be generated. Even if that were to be fixed (I tried modifying the .sh script, but I am unable to force it to not generate the debug version) this would still result in _install_python/lib/LibPose.lib to be generated instead of _install_python/lib/libLibPose.a.

I have no idea why adding --config=Release here has no effect: https://github.com/vlarsson/PoseLib/blob/bf48ad35ceeccab71d0b8bd7b34538c2e889a731/scripts/build_python_package.sh#L55-L58

My git bash might be broken in some way or use some old cmake. I do not really use it and maybe that's why my attempts at modifying scripts/build_python_package.sh have failed.

Anyways I can get it to work when I execute this .bat file. I can make a PR with this and put it into the scripts directory.

cmake -S . -B _build/ -DPYTHON_PACKAGE=ON -DCMAKE_INSTALL_PREFIX=_install
cmake --build _build/ --target install -j 8 --config=Release
cd _install/lib
copy PoseLib.lib libPoseLib.a
cd ../..
cmake --build _build/ --target pip-package
cmake --build _build/ --target install-pip-package

Also check you could build the package inside docker? See this PR #52.

I am not at all familiar with docker so I am afraid I won't be able to test that out.

[pablospe]

@ignacio-rocco. I was wondering if you've tried to compile in Windows. Not sure to understand why pip-package function is searching for .a instead of .lib in Windows, maybe you have an idea. Is DEPENDS ${LIBRARY_NAME} wrong in the following line? https://github.com/vlarsson/PoseLib/blob/master/pybind/CMakeLists.txt#L24

Also wonder if we should force -DCMAKE_BUILD_TYPE=Release here: https://github.com/vlarsson/PoseLib/blob/master/pybind/CMakeLists.txt#L18