robotology-superbuild icon indicating copy to clipboard operation
robotology-superbuild copied to clipboard

Python bindings do not work on Windows when installed from source since Python 3.8

Open traversaro opened this issue 2 years ago • 1 comments

In particular, for example import yarp is failing with:

>>> import yarp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\src\light_ws\rs\build\install\Lib\site-packages\yarp.py", line 15, in <module>
    import _yarp
ImportError: DLL load failed while importing _yarp: The specified module could not be found.
>>>

The reason is described in https://github.com/gazebosim/sdformat/pull/1165#issuecomment-1252967167, basically since Python 3.8:

New in version 3.8: Previous versions of CPython would resolve DLLs using the default behavior for the current process. This led to inconsistencies, such as only sometimes searching PATH or the current working directory, and OS functions such as AddDllDirectory having no effect.

In 3.8, the two primary ways DLLs are loaded now explicitly override the process-wide behavior to ensure consistency. See the porting notes for information on updating libraries.

traversaro avatar Sep 21 '22 09:09 traversaro

As suggested in https://github.com/gazebosim/sdformat/pull/1165#issuecomment-1252967167, running:

set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1

before using the bindings everything works as expected:

(robsublight) C:\src\light_ws\rs\build>set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1

(robsublight) C:\src\light_ws\rs\build>python
Python 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:30:19) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import yarp
>>> vec = yarp.Vector()
>>> vec.resize(10)
>>> vec.toString()
' 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000\t 0.000000'
>>>

traversaro avatar Sep 21 '22 09:09 traversaro

According to @giotherobot tests, set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1 is not working, so we are using:

import os
os.add_dll_directory(os.path.join(os.environ['ROBOTOLOGY_SUPERBUILD_INSTALL_PREFIX'], bin))

and we also documented it in https://github.com/robotology/robotology-superbuild/pull/1701 as a mitigation for this problem.

traversaro avatar Sep 05 '24 07:09 traversaro

Further mitigation is provided by https://github.com/robotology/idyntree/pull/1209 . Once that is merged, import idyntree will work out of the box, and also any subsequent call to import casadi, import bipedal_locomotion_framework or import yarp, as all the .dll of the superbuild are installed in the same folder. So technically speaking this issue will be solved once all libraries have something like https://github.com/robotology/idyntree/pull/1209, but in practice an alternative workaround to os.add_dll_directory(os.path.join(os.environ['ROBOTOLOGY_SUPERBUILD_INSTALL_PREFIX'], 'bin')) is to just add import idyntree.

traversaro avatar Sep 25 '24 12:09 traversaro