installation of mljar-supervised breaks sklearn in an Anaconda virtual environment: ImportError: DLL load failed: The parameter is incorrect.

[cpoptic]

Installation of mljar-supervised downgrades scikit-learn from 0.23 to 0.22, which does not import properly with the other dependencies.

python  -V
Python 3.7.9

conda -V
conda 4.8.4

pip install mljar-supervised

Installing collected packages: tensorflow-estimator, numpy, tensorboard, scipy, tensorflow, scikit-learn, mljar-supervised Attempting uninstall: tensorflow-estimator Found existing installation: tensorflow-estimator 2.3.0 Uninstalling tensorflow-estimator-2.3.0: Successfully uninstalled tensorflow-estimator-2.3.0 Attempting uninstall: numpy Found existing installation: numpy 1.18.5 Uninstalling numpy-1.18.5: Successfully uninstalled numpy-1.18.5 Attempting uninstall: tensorboard Found existing installation: tensorboard 2.3.0 Uninstalling tensorboard-2.3.0: Successfully uninstalled tensorboard-2.3.0 Attempting uninstall: scipy Found existing installation: scipy 1.5.2 Found existing installation: tensorflow 2.3.0 Uninstalling tensorflow-2.3.0: Successfully uninstalled tensorflow-2.3.0 Attempting uninstall: scikit-learn Found existing installation: scikit-learn 0.23.1 Uninstalling scikit-learn-0.23.1: Successfully uninstalled scikit-learn-0.23.1

visions 0.4.4 requires networkx>=2.4, but you'll have networkx 2.3 which is incompatible. prettierplot 0.1.2 requires scikit-learn>=0.22.2.post1, but you'll have scikit-learn 0.22.2 which is incompatible. mlmachine 0.1.4 requires scikit-learn>=0.22.2.post1, but you'll have scikit-learn 0.22.2 which is incompatible. lime 0.2.0.0 requires pillow==5.4.1, but you'll have pillow 7.2.0 which is incompatible. imbalanced-learn 0.7.0 requires scikit-learn>=0.23, but you'll have scikit-learn 0.22.2 which is incompatible. dbnd 0.27.10 requires configparser<3.6.0,>=3.5.0, but you'll have configparser 5.0.0 which is incompatible. dbnd 0.27.10 requires jinja2<2.11.0,>=2.10.1, but you'll have jinja2 2.11.2 which is incompatible. dbnd 0.27.10 requires pandas<1.0.0,>=0.17.1, but you'll have pandas 1.0.5 which is incompatible. dbnd 0.27.10 requires tzlocal<2.0.0,>=1.0.0, but you'll have tzlocal 2.1 which is incompatible. carefree-learn 0.1.2 requires scikit-learn>=0.23.1, but you'll have scikit-learn 0.22.2 which is incompatible. alphapy 2.5.0 requires scikit-learn>=0.23.1, but you'll have scikit-learn 0.22.2 which is incompatible. Successfully installed mljar-supervised-0.6.1 numpy-1.19.1 scikit-learn-0.22.2 scipy-1.4.1 tensorboard-2.2.2 tensorflow-2.2.0 tensorflow-estimator-2.2.0

C:\Users\user_name>python
Python 3.7.9 (default, Aug 31 2020, 17:10:11) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sklearn
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\ProgramData\Anaconda3\envs\my_env\lib\site-packages\sklemy_env\__init__.py", line 82, in <module>
    from .base import clone
  File "C:\ProgramData\Anaconda3\envs\my_env\lib\site-packages\sklemy_env\base.py", line 20, in <module>
    from .utils import _IS_32BIT
  File "C:\ProgramData\Anaconda3\envs\my_env\lib\site-packages\sklemy_env\utils\__init__.py", line 27, in <module>
    from .fixes import np_version
  File "C:\ProgramData\Anaconda3\envs\my_env\lib\site-packages\sklemy_env\utils\fixes.py", line 18, in <module>
    import scipy.stats
  File "C:\ProgramData\Anaconda3\envs\my_env\lib\site-packages\scipy\stats\__init__.py", line 384, in <module>
    from .stats import *
  File "C:\ProgramData\Anaconda3\envs\my_env\lib\site-packages\scipy\stats\stats.py", line 179, in <module>
    from scipy.spatial.distance import cdist
  File "C:\ProgramData\Anaconda3\envs\my_env\lib\site-packages\scipy\spatial\__init__.py", line 99, in <module>
    from .qhull import *
ImportError: DLL load failed: The parameter is incorrect.

But this sequence fixes it: pip uninstall mljar-supervised pip uninstall scikit-learn pip install -U scikit-learn pip install -U scipy

Fixes this by reinstalling scikit-learn 0.23.2 and scipy 1.5.2 and enables scikit-learn to be imported properly once again.

$ python Python 3.7.9 (default, Aug 31 2020, 17:10:11) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32 Type "help", "copyright", "credits" or "license" for more information.

>>> import sklearn
>>> import scipy
>>>

[pplonski]

@cpoptic thank you for reporting the problem. Probably we should update the scikit-learn version in requirements. Will look at it.

[pplonski]

@cpoptic I've updated the sklearn in requirements. Could you please install mljar-supervised in version 0.7.0? It should help. I'm closing the issue for now. If you will get still get errors please feel free to reopen.

[cpoptic]

I was able to solve this but the issue was your requirements.txt has scipy version 1.4, which is not compatible with scikit-learn 0.23.2 That version of sklearn requires scipy 1.5.2

The pip install mljar-supervised command still causes the aforementioned parameter is incorrect error when subsequently importing sklearn and scipy.

However, cloning the repo, and running python install setup.py ended up installing it correctly and the libraries are able to be imported without error. I'd recommend updating the requirements.txt from scipy 1.4 to 1.5, as that was the root of the error.

[pplonski]

I've updated scipy to 1.5.2 as you wrote.

Thank you for your help!

[pplonski]

@cpoptic you won't belive:

• I've updated scipy to 1.5.2
• Then I started to have errors in travis:

error: scipy 1.5.2 is installed but scipy==1.4.1; python_version >= "3" is required by {'tensorflow'}

• the old scipy is used in tests in TensorFlow https://github.com/tensorflow/tensorflow/issues/40884

I don't know what to do:

• scikit-learn 0.23.2 needs scipy 1.5.2

• tensorflow 2.2.0 needs scipy 1.4.1 (old dependency from tests)

• I was trying to upgrade tensorflow to 2.3.0 but it breaks the current Neural Network implementation (looks like 2.3.0 is not backward compatible with 2.2.0)

Please any idea what to do? (Maybe move NN implementation to PyTorch or just use MLP implementation from scikit-learn??? but this will require a lot of coding)

[cpoptic]

Ha welcome to the world of dependency hell!
I would not use scikit-learn for NN functionality when Keras+TF are much better. One possible solution is to wait for TF 2.4, which could fix the 2.3.0 version issue.

Although it appears TF 2.2.0 is working when installed alongside supervised:

>>> import tensorflow as tf
>>> tf.__version__
'2.2.0'
>>> import supervised
>>> import scipy
>>> import sklearn
>>> supervised.__version__
'0.7.1'
>>> scipy.__version__
'1.5.2'
>>> sklearn.__version__
'0.23.2'