signalflow icon indicating copy to clipboard operation
signalflow copied to clipboard

Cannor load DLL on WIndows

Open giohappy opened this issue 1 year ago • 19 comments

Python 3.11.9 on Windows 11. Signalflow fails when imprted.

This is the output when traced with dlltracer

>>> import dlltracer
>>> import sys
>>> with dlltracer.Trace(out=sys.stdout):
...     import signalflow
...
LoadLibrary C:\Windows\System32\kernel.appcore.dll
LoadLibrary C:\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow\signalflow.pyd
LoadLibrary C:\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow.libs\msvcp140-1ba8f0cd5a79973275397a73667e85ff.dll
LoadLibrary C:\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow.libs\sndfile-b0a52daf2e1c2a2cef4f74c8d536c443.dll
LoadLibrary C:\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow.libs\libfftw3f-3-3bdf804532d690cae74ca3fdac981db6.dll
Failed \Device\HarddiskVolume5\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow.libs\sndfile-b0a52daf2e1c2a2cef4f74c8d536c443.dll
Failed \Device\HarddiskVolume5\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow.libs\libfftw3f-3-3bdf804532d690cae74ca3fdac981db6.dll
Failed \Device\HarddiskVolume5\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow.libs\msvcp140-1ba8f0cd5a79973275397a73667e85ff.dll
Failed \Device\HarddiskVolume5\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow\signalflow.pyd
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "C:\Users\giohappy\AppData\Local\Programs\Python\Python311\Lib\site-packages\signalflow\__init__.py", line 20, in <module>
    from .signalflow import *
ImportError: DLL load failed while importing signalflow: The specified module could not be found.

The libraries are correctly installed where the tracer reports them.

By the way, inspecting the pyd library it looks like it links tp Python 3.10. Maybe this is the problem?

image

giohappy avatar Nov 29 '24 21:11 giohappy

I had another report of this which was eventually resolved… Do you happen to be running a 32-bit version of Python or otherwise in a 32-bit environment? The library is currently only built for 64-bit architectures, so I suspect this might be the issue.

ideoforms avatar Nov 30 '24 14:11 ideoforms

@ideoforms I'm on a 64 bit machine with Python 64 bit. I also hoped that was the reason, but no.

giohappy avatar Nov 30 '24 15:11 giohappy

This issue came back for me in a new venv. Looking at signalflow.pyd in Dependencies, signalflow's init.py is failing because the paths to the the .dlls don't seem to have resolved perhaps? I would have thought the .pyd should be resolving the paths to: \Lib\site-packages\signalflow.libs\name.dll

One quick resolution is simply copy the .dll's from signalflow.libs and place them in the \Lib\site-packages\signalflow directory, then: from signalflow import * should work.

Here you can see the three files in signalflow.pyd haven't been found, which is correct, they don't exist in the same directory as signalflow.pyd.

image

This is the error i get:

Traceback (most recent call last):
  File "C:\Users\skhoc\Documents\DANU\test.py", line 8, in <module>
    from signalflow import *
  File "C:\Users\skhoc\Documents\DANU\venv\Lib\site-packages\signalflow\__init__.py", line 21, in <module>
    from .signalflow import *
ImportError: DLL load failed while importing signalflow: The specified module could not be found.

Another way that should work is do this where you're importing signalflow:

import os

libs_dir = r"C:\Users\skhoc\Documents\DANU\venv\Lib\site-packages\signalflow.libs"
os.add_dll_directory(libs_dir)

# Try importing signalflow
try:
    from signalflow import *
    print("signalflow imported successfully")
except ImportError as e:
    print(f"ImportError: {e}")
except OSError as e:
    print(f"OSError: {e}")

hockinsk avatar Dec 01 '24 23:12 hockinsk

Actually, it makes no difference copying then to signalflow dir, it still doesn't find the three .dll.

hockinsk avatar Dec 02 '24 01:12 hockinsk

I thought I'd have a nose in signalflow.pyd hex to see what the dll paths were and found a lot of ascii error output related to JACK? Didn't expect to see that, i'm sure it's not related to this dll issue, but wasn't what I expected to find so sharing anyway.

I just removed all C++ redistributable, removed python, removed visual studio, installed python, installed signalflow but still errors the module can't be found.

image

hockinsk avatar Dec 02 '24 09:12 hockinsk

Rolling back to signalflow 0.4.4 works on same system. I notice there's a soundio .dll in soundflow.libs that doesn't exist in the 0.5.1 latest version, but assume this is handled differently now?

Then running pip install --upgrade signalflow to 0.5.1 appears to work too, it's just the fresh install directly to 0.5.1 has the issue? Weird.

PS C:\Users\skhoc\Documents\DANU> pip show signalflow
Name: signalflow
Version: 0.5.1
Summary: SignalFlow is a sound synthesis library designed for clear and concise expression of sonic ideas
Home-page:
Author: Daniel Jones
Author-email: [email protected]
License:
Location: C:\Users\skhoc\AppData\Local\Programs\Python\Python312\Lib\site-packages
Requires: numpy
Required-by:

hockinsk avatar Dec 02 '24 11:12 hockinsk

That's so strange. You're right, soundio.dll shouldn't be needed anymore. I'm travelling at the moment, but will take a look at the DLLs sometime when I'm back at my Windows computer next week.

ideoforms avatar Dec 05 '24 06:12 ideoforms

Great stuff. Probably looking at differences between 0.4.4 > 0.5.1 pip upgrade versus a fresh 0.5.1 install might show the issue because the upgrade path (using .whl instead of pip) seems to work.

hockinsk avatar Dec 07 '24 11:12 hockinsk

I bumped into the same issue now using a conda env that used python 3.10. I tried to perform @hockinsk's hack, but then noticed that v0.4.4 is cpython 12 only. Then I scratched the whole env and recreated it with python 3.12, and signalflow 0.5.1 (latest at the time of writing) and everything works. So, afaik it is not the soundio.dll, because on my system no version other than 0.5.1 was ever installed. There seems to be something in the cp310_amd64 wheel that has an issue.

balintlaczko avatar Jan 29 '25 14:01 balintlaczko

@balintlaczko So this was Windows/conda/py3.10? Do you still have that env? I'd be interested in some diags if you're still able to reproduce the problem, particularly pip show signalflow and the output of pip debug -v (which may be long but that's ok!).

ideoforms avatar Jan 29 '25 17:01 ideoforms

I I were to guess, it's JACK Audio Connection kit erroring while compiling signalflow.pyd and it's corrupted something, this is why upgrading from a working 0.4.4 to 0.5.1 seems to be fine, but installing directly to 0.5.1 simply isn't??

hockinsk avatar Jan 29 '25 20:01 hockinsk

Hey @ideoforms, I did not have the env anymore, but I reproduced it now, and will keep it around in case you want me to check things, just let me know! Here is how I reproduced it:

  1. conda create -n "sftest" python=3.10
  2. Get-Command pip (like which pip on Windows, checking that pip really is from the env)
  3. pip install --no-cache-dir signalflow (--no-cache-dir just to be sure => installs 0.5.1)
  4. python
Python 3.10.16 | packaged by Anaconda, Inc. | (main, Dec 11 2024, 16:19:12) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import signalflow as sf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\balin\.conda\envs\sftest\lib\site-packages\signalflow\__init__.py", line 20, in <module>
    from .signalflow import *
ImportError: DLL load failed while importing signalflow: The specified module could not be found.

For pip show signalflow I get

Name: signalflow
Version: 0.5.1
Summary: SignalFlow is a sound synthesis library designed for clear and concise expression of sonic ideas
Home-page:
Author: Daniel Jones
Author-email: [email protected]
License:
Location: c:\users\balin\.conda\envs\sftest\lib\site-packages
Requires: numpy
Required-by:

For pip debug -v I get

WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 23.3.2 from C:\Users\balin\AppData\Roaming\Python\Python310\site-packages\pip (python 3.10)
sys.version: 3.10.16 | packaged by Anaconda, Inc. | (main, Dec 11 2024, 16:19:12) [MSC v.1929 64 bit (AMD64)]
sys.executable: C:\Users\balin\.conda\envs\sftest\python.exe
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: cp1252
sys.platform: win32
sys.implementation:
  name: cpython
'cert' config value: Not specified
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): C:\Users\balin\AppData\Roaming\Python\Python310\site-packages\pip\_vendor\certifi\cacert.pem
pip._vendor.DEBUNDLED: False
vendored library versions:
  CacheControl==0.13.1
  colorama==0.4.6
  distlib==0.3.6
  distro==1.8.0
  msgpack==1.0.5
  packaging==21.3
  platformdirs==3.8.1
  pyparsing==3.1.0
  pyproject-hooks==1.0.0
  requests==2.31.0
  certifi==2023.07.22
  chardet==5.1.0
  idna==3.4
  urllib3==1.26.17
  rich==13.4.2 (Unable to locate actual module version, using vendor.txt specified version)
  pygments==2.15.1
  typing_extensions==4.7.1 (Unable to locate actual module version, using vendor.txt specified version)
  resolvelib==1.0.1
  setuptools==68.0.0 (Unable to locate actual module version, using vendor.txt specified version)
  six==1.16.0
  tenacity==8.2.2 (Unable to locate actual module version, using vendor.txt specified version)
  tomli==2.0.1
  truststore==0.8.0
  webencodings==0.5.1 (Unable to locate actual module version, using vendor.txt specified version)
Compatible tags: 36
  cp310-cp310-win_amd64
  cp310-abi3-win_amd64
  cp310-none-win_amd64
  cp39-abi3-win_amd64
  cp38-abi3-win_amd64
  cp37-abi3-win_amd64
  cp36-abi3-win_amd64
  cp35-abi3-win_amd64
  cp34-abi3-win_amd64
  cp33-abi3-win_amd64
  cp32-abi3-win_amd64
  py310-none-win_amd64
  py3-none-win_amd64
  py39-none-win_amd64
  py38-none-win_amd64
  py37-none-win_amd64
  py36-none-win_amd64
  py35-none-win_amd64
  py34-none-win_amd64
  py33-none-win_amd64
  py32-none-win_amd64
  py31-none-win_amd64
  py30-none-win_amd64
  cp310-none-any
  py310-none-any
  py3-none-any
  py39-none-any
  py38-none-any
  py37-none-any
  py36-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any

UPDATE: I purged (I think) all the other python versions on the computer that were outside conda. I scratched and recreated the sftest env exactly the same way as described above. Everything is the same (including the ImportError), except the output for pip debug -v:

WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 25.0 from C:\Users\balin\.conda\envs\sftest\lib\site-packages\pip (python 3.10)
sys.version: 3.10.16 | packaged by Anaconda, Inc. | (main, Dec 11 2024, 16:19:12) [MSC v.1929 64 bit (AMD64)]
sys.executable: C:\Users\balin\.conda\envs\sftest\python.exe
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: cp1252
sys.platform: win32
sys.implementation:
  name: cpython
'cert' config value: Not specified
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): C:\Users\balin\.conda\envs\sftest\lib\site-packages\pip\_vendor\certifi\cacert.pem
pip._vendor.DEBUNDLED: False
vendored library versions:
  CacheControl==0.14.1
  distlib==0.3.9
  distro==1.9.0
  msgpack==1.1.0
  packaging==24.2
  platformdirs==4.3.6
  pyproject-hooks==1.2.0
  requests==2.32.3
  certifi==2024.08.30
  idna==3.10
  urllib3==1.26.20
  rich==13.9.4 (Unable to locate actual module version, using vendor.txt specified version)
  pygments==2.18.0
  typing_extensions==4.12.2 (Unable to locate actual module version, using vendor.txt specified version)
  resolvelib==1.0.1
  setuptools==70.3.0 (Unable to locate actual module version, using vendor.txt specified version)
  tomli==2.2.1
  truststore==0.10.0
Compatible tags: 36
  cp310-cp310-win_amd64
  cp310-abi3-win_amd64
  cp310-none-win_amd64
  cp39-abi3-win_amd64
  cp38-abi3-win_amd64
  cp37-abi3-win_amd64
  cp36-abi3-win_amd64
  cp35-abi3-win_amd64
  cp34-abi3-win_amd64
  cp33-abi3-win_amd64
  cp32-abi3-win_amd64
  py310-none-win_amd64
  py3-none-win_amd64
  py39-none-win_amd64
  py38-none-win_amd64
  py37-none-win_amd64
  py36-none-win_amd64
  py35-none-win_amd64
  py34-none-win_amd64
  py33-none-win_amd64
  py32-none-win_amd64
  py31-none-win_amd64
  py30-none-win_amd64
  cp310-none-any
  py310-none-any
  py3-none-any
  py39-none-any
  py38-none-any
  py37-none-any
  py36-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any

balintlaczko avatar Jan 31 '25 09:01 balintlaczko

@hockinsk I cannot install v0.4.4 on python 3.10, and it seems like the release is only for Cpython 3.12? But if I use python 3.12 then the latest version v.5.0.1 also works without issues.

balintlaczko avatar Jan 31 '25 09:01 balintlaczko

Yeah, v0.4.4 was a Windows test release that only targeted 3.12. v0.5.1 is the first proper cross-Python release for Windows.

@balintlaczko Do you think you would be able to try running this utility on the different signalflow .pyd files in your 3.10 vs 3.12 conda envs? (A .pyd file is really just a DLL in disguise on Windows). This might shed light on which DLL it's failing to find.

@hockinsk The warning messages about JACK when you look inside the binary are simply because SignalFlow now compiles with miniaudio as an include, so these are benign and expected.

ideoforms avatar Jan 31 '25 10:01 ideoforms

@ideoforms sure thing! This is what it shows for the failing env ("sftest") with python 3.10

Image

And this is what I get in the env where it works ("pixasonics") with python 3.12

Image

Not sure how to interpret this, but hope it helps!

balintlaczko avatar Jan 31 '25 13:01 balintlaczko

Interesting! The 3.10 build seems to be looking for python39.dll, which is definitely not right. I'll investigate whether this is a problem with things not properly getting configured or cleaned up on the build machine.

Thanks so much for your help @balintlaczko, I will try to get a new build out as soon as I have the opportunity.

ideoforms avatar Jan 31 '25 13:01 ideoforms

Thanks a lot!!! (But as for my case, no rush! I'm fine using 3.12 in the env.)

balintlaczko avatar Jan 31 '25 14:01 balintlaczko

Thanks for all your help with this. I tracked this down to a silly issue in the build process, in which it sometimes bundled the wrong cached version of the .pyd file. 🤪

This should now be fixed!

Could you please try installing the latest (0.5.2, Windows-only) release and let me know if this resolves the issue @balintlaczko @hockinsk @giohappy?

ideoforms avatar Feb 10 '25 09:02 ideoforms

Thanks for the fix, @ideoforms, this solved it for me! In the same "sftest" env that was failing before under python 3.10 I uninstalled the previous signalflow 0.5.1 and installed the new 0.5.2, and then imported signalflow without the dll error. Nice!

balintlaczko avatar Feb 10 '25 10:02 balintlaczko