conda-forge.github.io
conda-forge.github.io copied to clipboard
conda-forge
File conflict between llvm-openmp (and old openmp) and intel-openmp
Conda-forge documentation
- [X] I could not solve my problem using the conda-forge documentation.
Installed packages
# packages in environment at D:\miniforge3:
#
# Name Version Build Channel
anyio 3.5.0 py39hcbf5309_0 conda-forge
argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py39hb82d6ee_1 conda-forge
asttokens 2.0.5 pyhd8ed1ab_0 conda-forge
attrs 21.4.0 pyhd8ed1ab_0 conda-forge
babel 2.9.1 pyh44b312d_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
black 22.1.0 pyhd8ed1ab_0 conda-forge
blas 2.113 mkl conda-forge
blas-devel 3.9.0 13_win64_mkl conda-forge
bleach 4.1.0 pyhd8ed1ab_0 conda-forge
brotli 1.0.9 h8ffe710_6 conda-forge
brotli-bin 1.0.9 h8ffe710_6 conda-forge
brotli-python 1.0.9 py39h415ef7b_6 conda-forge
brotlipy 0.7.0 py39hb82d6ee_1003 conda-forge
bzip2 1.0.8 h8ffe710_4 conda-forge
ca-certificates 2021.10.8 h5b45459_0 conda-forge
certifi 2021.10.8 py39hcbf5309_1 conda-forge
cffi 1.15.0 py39h0878f49_0 conda-forge
charset-normalizer 2.0.11 pyhd8ed1ab_0 conda-forge
click 8.0.3 py39hcbf5309_1 conda-forge
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
conda 4.11.0 py39hcbf5309_0 conda-forge
conda-package-handling 1.7.3 py39hb3671d1_1 conda-forge
cpuonly 2.0 0 pytorch
cryptography 36.0.1 py39h7bc7c5c_0 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
dash 2.1.0 pyhd8ed1ab_0 conda-forge
dataclasses 0.8 pyhc8e2a94_3 conda-forge
debugpy 1.5.1 py39h415ef7b_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
entrypoints 0.3 pyhd8ed1ab_1003 conda-forge
executing 0.8.2 pyhd8ed1ab_0 conda-forge
flask 2.0.2 pyhd8ed1ab_0 conda-forge
flask-compress 1.10.1 pyhd8ed1ab_0 conda-forge
flit-core 3.6.0 pyhd8ed1ab_0 conda-forge
fonttools 4.29.1 py39hb82d6ee_0 conda-forge
freetype 2.10.4 h546665d_1 conda-forge
icu 68.2 h0e60522_0 conda-forge
idna 3.3 pyhd8ed1ab_0 conda-forge
importlib-metadata 4.10.1 py39hcbf5309_0 conda-forge
importlib_resources 5.4.0 pyhd8ed1ab_0 conda-forge
intel-openmp 2022.0.0 h57928b3_3663 conda-forge
ipykernel 6.7.0 py39h832f523_0 conda-forge
ipython 8.0.1 py39hcbf5309_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
ipywidgets 7.6.5 pyhd8ed1ab_0 conda-forge
itsdangerous 2.0.1 pyhd8ed1ab_0 conda-forge
jbig 2.1 h8d14728_2003 conda-forge
jedi 0.18.1 py39hcbf5309_0 conda-forge
jinja2 3.0.3 pyhd8ed1ab_0 conda-forge
jpeg 9e h8ffe710_0 conda-forge
json5 0.9.5 pyh9f0ad1d_0 conda-forge
jsonschema 4.4.0 pyhd8ed1ab_0 conda-forge
jupyter 1.0.0 py39hcbf5309_7 conda-forge
jupyter_client 7.1.2 pyhd8ed1ab_0 conda-forge
jupyter_console 6.4.0 pyhd8ed1ab_0 conda-forge
jupyter_core 4.9.1 py39hcbf5309_1 conda-forge
jupyter_server 1.13.4 pyhd8ed1ab_0 conda-forge
jupyterlab 3.2.8 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge
jupyterlab_server 2.10.3 pyhd8ed1ab_0 conda-forge
jupyterlab_widgets 1.0.2 pyhd8ed1ab_0 conda-forge
kiwisolver 1.3.2 py39h2e07f2f_1 conda-forge
lcms2 2.12 h2a16943_0 conda-forge
lerc 3.0 h0e60522_0 conda-forge
libblas 3.9.0 13_win64_mkl conda-forge
libbrotlicommon 1.0.9 h8ffe710_6 conda-forge
libbrotlidec 1.0.9 h8ffe710_6 conda-forge
libbrotlienc 1.0.9 h8ffe710_6 conda-forge
libcblas 3.9.0 13_win64_mkl conda-forge
libclang 11.1.0 default_h5c34c98_1 conda-forge
libdeflate 1.8 h8ffe710_0 conda-forge
libffi 3.4.2 h8ffe710_5 conda-forge
liblapack 3.9.0 13_win64_mkl conda-forge
liblapacke 3.9.0 13_win64_mkl conda-forge
libpng 1.6.37 h1d00b33_2 conda-forge
libsodium 1.0.18 h8d14728_1 conda-forge
libtiff 4.3.0 hd413186_2 conda-forge
libuv 1.43.0 h8ffe710_0 conda-forge
libzlib 1.2.11 h8ffe710_1013 conda-forge
lz4-c 1.9.3 h8ffe710_1 conda-forge
m2w64-gcc-libgfortran 5.3.0 6 conda-forge
m2w64-gcc-libs 5.3.0 7 conda-forge
m2w64-gcc-libs-core 5.3.0 7 conda-forge
m2w64-gmp 6.1.0 2 conda-forge
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge
markupsafe 2.0.1 py39hb82d6ee_1 conda-forge
matplotlib 3.5.1 py39hcbf5309_0 conda-forge
matplotlib-base 3.5.1 py39h581301d_0 conda-forge
matplotlib-inline 0.1.3 pyhd8ed1ab_0 conda-forge
menuinst 1.4.18 py39hcbf5309_1 conda-forge
miniforge_console_shortcut 2.0 h57928b3_1 conda-forge
mistune 0.8.4 py39hb82d6ee_1005 conda-forge
mkl 2022.0.0 h0e2418a_796 conda-forge
mkl-devel 2022.0.0 h57928b3_797 conda-forge
mkl-include 2022.0.0 h0e2418a_796 conda-forge
msys2-conda-epoch 20160418 1 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
mypy_extensions 0.4.3 py39hcbf5309_4 conda-forge
nbclassic 0.3.5 pyhd8ed1ab_0 conda-forge
nbclient 0.5.10 pyhd8ed1ab_1 conda-forge
nbconvert 6.4.1 py39hcbf5309_0 conda-forge
nbformat 5.1.3 pyhd8ed1ab_0 conda-forge
nest-asyncio 1.5.4 pyhd8ed1ab_0 conda-forge
notebook 6.4.8 pyha770c72_0 conda-forge
numpy 1.22.1 py39h6331f09_0 conda-forge
olefile 0.46 pyh9f0ad1d_1 conda-forge
openjpeg 2.4.0 hb211442_1 conda-forge
openssl 1.1.1l h8ffe710_0 conda-forge
packaging 21.3 pyhd8ed1ab_0 conda-forge
pandoc 2.17.1.1 h57928b3_0 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pathspec 0.9.0 pyhd8ed1ab_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 8.4.0 py39h916092e_0 conda-forge
pip 22.0.2 pyhd8ed1ab_0 conda-forge
platformdirs 2.3.0 pyhd8ed1ab_0 conda-forge
plotly 5.5.0 pyhd8ed1ab_0 conda-forge
prometheus_client 0.13.1 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.26 pyha770c72_0 conda-forge
prompt_toolkit 3.0.26 hd8ed1ab_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pycosat 0.6.3 py39hb82d6ee_1009 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.11.2 pyhd8ed1ab_0 conda-forge
pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.7 pyhd8ed1ab_0 conda-forge
pyqt 5.12.3 py39hcbf5309_8 conda-forge
pyqt-impl 5.12.3 py39h415ef7b_8 conda-forge
pyqt5-sip 4.19.18 py39h415ef7b_8 conda-forge
pyqtchart 5.12 py39h415ef7b_8 conda-forge
pyqtwebengine 5.12.1 py39h415ef7b_8 conda-forge
pyrsistent 0.18.1 py39hb82d6ee_0 conda-forge
pysocks 1.7.1 py39hcbf5309_4 conda-forge
python 3.9.10 h9a09f29_2_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.9 2_cp39 conda-forge
pytorch 1.10.2 py3.9_cpu_0 pytorch
pytorch-mutex 1.0 cpu pytorch
pytz 2021.3 pyhd8ed1ab_0 conda-forge
pywin32 303 py39hb82d6ee_0 conda-forge
pywinpty 2.0.1 py39h99910a6_0 conda-forge
pyzmq 22.3.0 py39he46f08e_1 conda-forge
qt 5.12.9 h5909a2a_4 conda-forge
qtconsole 5.2.2 pyhd8ed1ab_1 conda-forge
qtconsole-base 5.2.2 pyhd8ed1ab_1 conda-forge
qtpy 2.0.0 pyhd8ed1ab_0 conda-forge
requests 2.27.1 pyhd8ed1ab_0 conda-forge
ruamel_yaml 0.15.80 py39hb82d6ee_1006 conda-forge
send2trash 1.8.0 pyhd8ed1ab_0 conda-forge
setuptools 60.6.0 py39hcbf5309_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sniffio 1.2.0 py39hcbf5309_2 conda-forge
sqlite 3.37.0 h8ffe710_0 conda-forge
stack_data 0.1.4 pyhd8ed1ab_0 conda-forge
tbb 2021.5.0 h2d74725_0 conda-forge
tenacity 8.0.1 pyhd8ed1ab_0 conda-forge
terminado 0.13.1 py39hcbf5309_0 conda-forge
testpath 0.5.0 pyhd8ed1ab_0 conda-forge
tk 8.6.11 h8ffe710_1 conda-forge
tomli 2.0.0 pyhd8ed1ab_1 conda-forge
tornado 6.1 py39hb82d6ee_2 conda-forge
tqdm 4.62.3 pyhd8ed1ab_0 conda-forge
traitlets 5.1.1 pyhd8ed1ab_0 conda-forge
typed-ast 1.5.2 py39hb82d6ee_0 conda-forge
typing_extensions 4.0.1 pyha770c72_0 conda-forge
tzdata 2021e he74cb21_0 conda-forge
ucrt 10.0.20348.0 h57928b3_0 conda-forge
unicodedata2 14.0.0 py39hb82d6ee_0 conda-forge
urllib3 1.26.8 pyhd8ed1ab_1 conda-forge
vc 14.2 hb210afc_6 conda-forge
vs2015_runtime 14.29.30037 h902a5da_6 conda-forge
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 py_1 conda-forge
websocket-client 1.2.3 pyhd8ed1ab_0 conda-forge
werkzeug 2.0.2 pyhd8ed1ab_0 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
widgetsnbextension 3.5.2 py39hcbf5309_1 conda-forge
win_inet_pton 1.1.0 py39hcbf5309_3 conda-forge
winpty 0.4.3 4 conda-forge
xz 5.2.5 h62dcd97_1 conda-forge
yaml 0.2.5 h8ffe710_2 conda-forge
zeromq 4.3.4 h0e60522_1 conda-forge
zipp 3.7.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.11 h8ffe710_1013 conda-forge
zstd 1.5.2 h6255e5f_0 conda-forge
Environment info
active environment : None
user config file : C:\Users\yangy\.condarc
populated config files : D:\miniforge3\.condarc
C:\Users\yangy\.condarc
conda version : 4.11.0
conda-build version : not installed
python version : 3.9.10.final.0
virtual packages : __win=0=0
__archspec=1=x86_64
base environment : D:\miniforge3 (writable)
conda av data dir : D:\miniforge3\etc\conda
conda av metadata url : None
channel URLs : https://conda.anaconda.org/conda-forge/win-64
https://conda.anaconda.org/conda-forge/noarch
https://conda.anaconda.org/pytorch/win-64
https://conda.anaconda.org/pytorch/noarch
package cache : D:\miniforge3\pkgs
C:\Users\yangy\.conda\pkgs
C:\Users\yangy\AppData\Local\conda\conda\pkgs
envs directories : D:\miniforge3\envs
C:\Users\yangy\.conda\envs
C:\Users\yangy\AppData\Local\conda\conda\envs
platform : win-64
user-agent : conda/4.11.0 requests/2.27.1 CPython/3.9.10 Windows/10 Windows/10.0.19044
administrator : False
netrc file : None
offline mode : False
Issue
I recently find the file conflict of openmp 5.0.0 and intel-openmp
PS D:\miniforge3\pkgs> rg -g *.json libiomp
openmp-5.0.0-vc14_1\info\paths.json
4: "_path": "Library/bin/libiomp5md.dll",
28: "_path": "Library/lib/libiomp5md.dll",
intel-openmp-2022.0.0-h57928b3_3663\info\paths.json
4: "_path": "Library/bin/libiomp5md.dll",
10: "_path": "Library/bin/libiomp5md.pdb",
16: "_path": "Library/bin/libiompstubs5md.dll",
They both contain libiomp5md.dll and this file will be overwritten by the latest installed package without warning.
It causes DLL load error on windows for packages depending on intel-openmp.
The conflict packages URLs:
- https://conda.anaconda.org/conda-forge/win-64/intel-openmp-2022.0.0-h57928b3_3663.tar.bz2
- https://conda.anaconda.org/conda-forge/win-64/openmp-5.0.0-vc14_1.tar.bz2
openmp-5.0.0 is almost 4 years old; the current version is 8.0.1. On the face of it, it doesn't seem surprising that two different implementations of the same api are colliding.
What is the actual problem that you are encountering? In other words, what circumstances lead to these two packages being installed in the same environment?
The following packages depend on libflang 5.0.0
- scip-8.0.0-h6aab56e_0
- ipopt-3.14.4-hf6be2e5_0
- casadi-3.5.5-py39h654df56_7
libflang 5.0.0 depends on openmp 5.0.0.
pytorch depends on intel-openmp.
If I install ipopt and pytorch in the same environment, libiomp5md.dll will conflict. openmp overwrites libiomp5md.dll during installation.
Dependencies GUI says that libiomp5md.dll from openmp lacks some function so some DLLs of pytorch cannot be loaded correctly.
Hm. The two packages scip and casadi depend on flang only via ipopt, as far as I can see, so let's focus on that and see where we can get.
Looking around a bit further, I think you just accidentally stepped into a right royal mess. This is, by and large, because the fortran compiler flang used to be outside of llvm, but now there is a new (different) flang inside of llvm, but that one isn't super mature yet. I think the best way forward is to drive the following two issues/PRs and the associated migrator to get the flang situation, particularly on Windows improved:
- conda-forge/openblas-feedstock#115
- conda-forge/conda-forge.github.io#1269
Can conda forge packages compiled by Intel fortran or gfortran on Windows? Flang 5.0.0 is quite old. This PR will upgrade flang version but is still open. https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/1359
I find that libiomp5md.dll from llvm-openmp 12.0.1 is still incompatible with the one provided by intel-openmp, so upgrading flang will not solve this problem.
Two URLs:
- https://conda.anaconda.org/conda-forge/win-64/intel-openmp-2022.0.0-h57928b3_3663.tar.bz2
- https://conda.anaconda.org/conda-forge/win-64/llvm-openmp-12.0.1-h2d74725_1.tar.bz2
I think it is a serious problem because mkl depends on intel-openmp and llvm-openmp is also depended by lots of packages. Silent overwrite of incompatible DLL files with the same name is problematic and leads to confusing loading errors.
Haven't read through the discussion in all detail, but the following is probably relevant: https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/1359
This PR will upgrade flang version but is still open. conda-forge/conda-forge-pinning-feedstock#1359
Sorry I overlooked that you had already mentioned this.
I find that
libiomp5md.dllfromllvm-openmp12.0.1 is still incompatible with the one provided byintel-openmp, so upgrading flang will not solve this problem. Two URLs: [...] I think it is a serious problem becausemkldepends onintel-openmpandllvm-openmpis also depended by lots of packages. Silent overwrite of incompatible DLL files with the same name is problematic and leads to confusing loading errors.
@conda-forge/openmp @conda-forge/intel_repack The split between llvm-openmp & intel-openmp on windows is getting more and more painful. For example, it's currently leading to pytorch (depending on intel-openmp) being broken by pillow adding a new dependency (libimagequant, which depends on llvm-openmp)
PRs are always welcome to add windows builds to pytorch.
Of course**. But isn't the underlying issue here that conda-forge's mkl depends on intel-openmp?
** took this as an opportunity to pick up the discussion about the special ops queue again.
Of course**. But isn't the underlying issue here that conda-forge's mkl depends on intel-openmp?
That's because of pytorch. If we have a pytorch windows build, then we can make mkl depend on llvm-openmp. pytorch upstream seems to be linking to the vcomp interface in libiomp5md.dll which is not present in llvm-openmp, but we can avoid that if we have a build on windows.
Cross-reference to an old but related issue for people that may find that issue on search engines: https://github.com/conda-forge/conda-forge.github.io/issues/727 . fyi @FabioBergonti
