conda-forge
Unable to use MinGW instead of MSVC for building packages on Windows
Your question:
Hello!
I am trying to contribute the Lets-Plot package to conda-forge. That project is based on Kotlin Native with Python API.
The issue I am facing is that conda-build defaults to using MSVC on Windows, but the library needs to be built with MinGW, because Kotlin Native supports only MinGW as a build target for the Windows platform.
Even after configuring meta.yaml, bld.bat, and setup.py to explicitly call MinGW, conda-build continues to invoke MSVC.
Steps taken:
- Modified
bld.batto configure MinGW. - Adjusted
setup.pyto link to MinGW binaries. - Set appropriate environment variables.
- Despite these modifications,
conda-buildcontinues to use MSVC.
Is there an official way to force conda-build to use MinGW for Windows builds? Or are there any known workarounds or future plans to support this?
Do you have a PR on staged-recipes? One need not have a fully worked out recipe to start a draft and let others see how it fails.
All R packages build through MinGW, so maybe have a look at their recipes. Recently we've been transitioning to UCRT, so some of the recipes might be in flux. For example, here's a recent submission without UCRT.
I believe it's mostly a matter of using compiler('m2w64_cxx') (for example) instead of compiler('cxx').
Hello! Thank you for the reply!
Do you have a PR on staged-recipes?
I have just made one:
https://github.com/conda-forge/staged-recipes/pull/27974
For example, here's https://github.com/conda-forge/staged-recipes/pull/27938.
I tried many things from that repository, but they didn't help. I also noticed that there is a problem with the CI build of that PR on Windows. However, I am not sure if it’s caused by MinGW problems.
I believe it's mostly a matter of using compiler('m2w64_cxx') (for example) instead of compiler('cxx').
When I try something like {{ compiler('m2w64_cxx') }} instead of {{ compiler('cxx') }}, conda starts searching for the m2w64_cxx_win-64 package, which doesn't exist. This issue occurs with any attempt to use m2w64_... instead of standard compiler options. I even tried to create a m2w64_cxx_win-64 package that links to the original m2w64_cxx package locally, but it didn’t help override the MSVC call.
I tried to build conda packages locally on my Windows machine by running the command (according to the documentation):
conda build recipes\lets-plot\ -m .\.ci_support\win64.yaml
I also tried running just conda build:
conda build recipes\lets-plot\
But there was no difference.
Build machine info:
Edition Windows 11 Pro
Version 23H2
Installed on 18.01.2023
OS build 22631.4317
Experience Windows Feature Experience Pack 1000.22700.1041.0
Conda build environment info:
active environment : test-conda-build
active env location : D:\Util\miniconda3\envs\test-conda-build
shell level : 2
user config file : C:\Users\Valentin Dovidaytis\.condarc
populated config files :
conda version : 24.7.1
conda-build version : not installed
python version : 3.12.4.final.0
solver : libmamba (default)
virtual packages : __archspec=1=skylake
__conda=24.7.1=0
__win=0=0
base environment : D:\Util\miniconda3 (writable)
conda av data dir : D:\Util\miniconda3\etc\conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : D:\Util\miniconda3\pkgs
C:\Users\Valentin Dovidaytis\.conda\pkgs
C:\Users\Valentin Dovidaytis\AppData\Local\conda\conda\pkgs
envs directories : D:\Util\miniconda3\envs
C:\Users\Valentin Dovidaytis\.conda\envs
C:\Users\Valentin Dovidaytis\AppData\Local\conda\conda\envs
platform : win-64
user-agent : conda/24.7.1 requests/2.32.3 CPython/3.12.4 Windows/11 Windows/10.0.22631 solver/libmamba conda-libmamba-solver/24.7.0 libmambapy/1.5.8 aau/0.4.4 c/. s/. e/.
administrator : False
netrc file : None
offline mode : False
Conda environment packages:
# packages in environment at D:\Util\miniconda3\envs\test-conda-build:
#
# Name Version Build Channel
archspec 0.2.3 pyhd8ed1ab_0 conda-forge
attrs 24.2.0 pyh71513ae_0 conda-forge
beautifulsoup4 4.12.3 pyha770c72_0 conda-forge
boltons 24.0.0 pyhd8ed1ab_0 conda-forge
brotli-python 1.1.0 py312h275cf98_2 conda-forge
bzip2 1.0.8 h2466b09_7 conda-forge
ca-certificates 2024.8.30 h56e8100_0 conda-forge
certifi 2024.8.30 pyhd8ed1ab_0 conda-forge
cffi 1.17.1 py312h4389bb4_0 conda-forge
chardet 5.2.0 py312h2e8e312_2 conda-forge
charset-normalizer 3.4.0 pyhd8ed1ab_0 conda-forge
click 8.1.7 win_pyh7428d3b_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
conda 24.9.2 py312h2e8e312_0 conda-forge
conda-build 24.9.0 py312h2e8e312_0 conda-forge
conda-index 0.5.0 pyhd8ed1ab_0 conda-forge
conda-libmamba-solver 24.9.0 pyhd8ed1ab_0 conda-forge
conda-package-handling 2.4.0 pyh7900ff3_0 conda-forge
conda-package-streaming 0.11.0 pyhd8ed1ab_0 conda-forge
distro 1.9.0 pyhd8ed1ab_0 conda-forge
filelock 3.16.1 pyhd8ed1ab_0 conda-forge
fmt 11.0.2 h7f575de_0 conda-forge
frozendict 2.4.6 py312h4389bb4_0 conda-forge
h2 4.1.0 pyhd8ed1ab_0 conda-forge
hpack 4.0.0 pyh9f0ad1d_0 conda-forge
hyperframe 6.0.1 pyhd8ed1ab_0 conda-forge
idna 3.10 pyhd8ed1ab_0 conda-forge
importlib_resources 6.4.5 pyhd8ed1ab_0 conda-forge
jinja2 3.1.4 pyhd8ed1ab_0 conda-forge
jsonpatch 1.33 pyhd8ed1ab_0 conda-forge
jsonpointer 3.0.0 py312h2e8e312_1 conda-forge
jsonschema 4.23.0 pyhd8ed1ab_0 conda-forge
jsonschema-specifications 2024.10.1 pyhd8ed1ab_0 conda-forge
krb5 1.21.3 hdf4eb48_0 conda-forge
libarchive 3.7.4 haf234dc_0 conda-forge
libcurl 8.10.1 h1ee3ff0_0 conda-forge
libexpat 2.6.3 he0c23c2_0 conda-forge
libffi 3.4.2 h8ffe710_5 conda-forge
libiconv 1.17 hcfcfb64_2 conda-forge
liblief 0.14.1 he0c23c2_2 conda-forge
libmamba 1.5.10 h81425b0_1 conda-forge
libmambapy 1.5.10 py312h643a1bd_1 conda-forge
libsolv 0.7.30 hbb528cf_0 conda-forge
libsqlite 3.47.0 h2466b09_0 conda-forge
libssh2 1.11.0 h7dfc565_0 conda-forge
libxml2 2.12.7 h0f24e4e_4 conda-forge
libzlib 1.3.1 h2466b09_2 conda-forge
lz4-c 1.9.4 hcfcfb64_0 conda-forge
lzo 2.10 hcfcfb64_1001 conda-forge
m2-conda-epoch 20230914 0_x86_64 conda-forge
m2-msys2-runtime 3.4.9.1 hd8ed1ab_4 conda-forge
m2-patch 2.7.6.2 hd8ed1ab_4 conda-forge
markupsafe 3.0.2 py312h31fea79_0 conda-forge
menuinst 2.1.2 py312h275cf98_1 conda-forge
more-itertools 10.5.0 pyhd8ed1ab_0 conda-forge
openssl 3.3.2 h2466b09_0 conda-forge
packaging 24.1 pyhd8ed1ab_0 conda-forge
pip 24.2 pyh8b19718_1 conda-forge
pkginfo 1.11.2 pyhd8ed1ab_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1 conda-forge
platformdirs 4.3.6 pyhd8ed1ab_0 conda-forge
pluggy 1.5.0 pyhd8ed1ab_0 conda-forge
psutil 6.0.0 py312h4389bb4_2 conda-forge
py-lief 0.14.1 py312h275cf98_2 conda-forge
pybind11-abi 4 hd8ed1ab_3 conda-forge
pycosat 0.6.6 py312he70551f_0 conda-forge
pycparser 2.22 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 pyh0701188_6 conda-forge
python 3.12.7 hce54a09_0_cpython conda-forge
python-libarchive-c 5.1 py312h2e8e312_1 conda-forge
python_abi 3.12 5_cp312 conda-forge
pytz 2024.2 pyhd8ed1ab_0 conda-forge
pyyaml 6.0.2 py312h4389bb4_1 conda-forge
referencing 0.35.1 pyhd8ed1ab_0 conda-forge
reproc 14.2.4.post0 hcfcfb64_1 conda-forge
reproc-cpp 14.2.4.post0 h63175ca_1 conda-forge
requests 2.32.3 pyhd8ed1ab_0 conda-forge
ripgrep 14.1.0 ha073cba_1 conda-forge
rpds-py 0.20.0 py312h2615798_1 conda-forge
ruamel.yaml 0.18.6 py312h4389bb4_1 conda-forge
ruamel.yaml.clib 0.2.8 py312h4389bb4_1 conda-forge
setuptools 75.1.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.5 pyhd8ed1ab_1 conda-forge
tk 8.6.13 h5226925_1 conda-forge
tqdm 4.66.5 pyhd8ed1ab_0 conda-forge
truststore 0.9.2 pyhd8ed1ab_0 conda-forge
tzdata 2024b hc8b5060_0 conda-forge
ucrt 10.0.22621.0 h57928b3_1 conda-forge
urllib3 2.2.3 pyhd8ed1ab_0 conda-forge
vc 14.3 h8a93ad2_22 conda-forge
vc14_runtime 14.40.33810 hcc2c482_22 conda-forge
vs2015_runtime 14.40.33810 h3bf8584_22 conda-forge
wheel 0.44.0 pyhd8ed1ab_0 conda-forge
win_inet_pton 1.1.0 pyh7428d3b_7 conda-forge
xz 5.2.6 h8d14728_0 conda-forge
yaml 0.2.5 h8ffe710_2 conda-forge
yaml-cpp 0.8.0 h63175ca_0 conda-forge
zipp 3.20.2 pyhd8ed1ab_0 conda-forge
zstandard 0.23.0 py312h7606c53_1 conda-forge
zstd 1.5.6 h0ea2cb4_0 conda-forge
I have just made one:
As I expected, the CI build on the Windows server crashed at the extension compilation step because MSVC was called instead of the MinGW compiler:
creating build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -ID:\bld\lets-plot_1729682723651\work\python-extension\build\bin\native\releaseStatic -ID:\bld\lets-plot_1729682723651\_h_env\include -ID:\bld\lets-plot_1729682723651\_h_env\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -ID:\bld\lets-plot_1729682723651\_h_env\Library\include -ID:\bld\lets-plot_1729682723651\_h_env\Library\include /d1trimfile:D:\bld\lets-plot_1729682723651\work /TcD:\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.c /Fobuild\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.obj
lets_plot_kotlin_bridge.c
D:\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.c(47): warning C4090: 'function': different 'const' qualifiers
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:D:\bld\lets-plot_1729682723651\work\python-extension\build\bin\native\releaseStatic /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\libs /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\PCbuild\amd64 "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\Library\lib /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\Library\lib lets_plot_python_extension.lib stdc++.lib /EXPORT:PyInit_lets_plot_kotlin_bridge build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.obj /OUT:build\lib.win-amd64-cpython-311\lets_plot_kotlin_bridge.cp311-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.cp311-win_amd64.lib -static-libgcc -static -lbcrypt -lpthread -lz
LINK : warning LNK4044: unrecognized option '/static-libgcc'; ignored
LINK : warning LNK4044: unrecognized option '/static'; ignored
LINK : warning LNK4044: unrecognized option '/lbcrypt'; ignored
LINK : warning LNK4044: unrecognized option '/lpthread'; ignored
LINK : warning LNK4044: unrecognized option '/lz'; ignored
LINK : fatal error LNK1181: cannot open input file 'lets_plot_python_extension.lib'
error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX64\\x64\\link.exe' failed with exit code 1181
error: subprocess-exited-with-error
Building wheel for lets-plot (setup.py): finished with status 'error'
The compiler('m2w64_cxx') setting was used in the recipe, but it was ignored by conda.
What steps should I try next to fix the Windows build?
I believe the issue is that those definitions are derived from conda_build_config.yamlin the conda-forge-pinning-feedstock repo. Those still use the old definitions because this migration https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/migrations/r-base44_and_m2w64-ucrt.yaml is still not completed. If I understand correctly, you need to manually enter the new definitions from that migration in your project's conda_build_config.yaml.
I believe the issue is that those definitions are derived from conda_build_config.yamlin the conda-forge-pinning-feedstock repo. Those still use the old definitions because this migration https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/migrations/r-base44_and_m2w64-ucrt.yaml is still not completed. If I understand correctly, you need to manually enter the new definitions from that migration in your project's conda_build_config.yaml.
Thank you for the answer! Unfortunately, it didn’t work. It seems that we need to wait for official MinGW support from the conda-forge team.
