maturin icon indicating copy to clipboard operation
maturin copied to clipboard

Published 20 hours ago • verified publisher icon PyO3

Maturin Fails to x-compile for target `--target i686-pc-windows-msvc`

Open peeksake opened this issue 1 year ago • 2 comments

Bug Description

error: linking with `lld-link` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/Users/peek/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/Users/peek/dev/work/ngc-py-rust/.venv/bin:/Users/peek/.rd/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/Library/Frameworks/Python.framework/Versions/3.9/bin:/Users/peek/.rd/bin:/Users/peek/.local/bin:/Users/peek/.cargo/bin:/opt/homebrew/bin:/Users/peek/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Users/peek/Library/Caches/cargo-xwin:/opt/homebrew/opt/llvm/bin" VSLANG="1033" "lld-link" "-flavor" "link" "/DEF:/var/folders/4j/ngr_tv3x4412d67vc01xmjsh0000gp/T/rustcrtVXXJ/lib.def" "/NOLOGO" "/LARGEADDRESSAWARE" "/SAFESEH" "/var/folders/4j/ngr_tv3x4412d67vc01xmjsh0000gp/T/rustcrtVXXJ/symbols.o" "/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/deps/ngc_rtc.ngc_rtc.f35e48bc522c81cd-cgu.0.rcgu.o" "/LIBPATH:/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/deps" "/LIBPATH:/Users/peek/dev/work/ngc-py-rust/target/release/deps" "/LIBPATH:/Users/peek/Library/Caches/cargo-xwin/xwin/crt/lib/x86" "/LIBPATH:/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/lib/um/x86" "/LIBPATH:/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/lib/ucrt/x86" "/LIBPATH:/Users/peek/.cargo/registry/src/index.crates.io-6f17d22bba15001f/windows_i686_msvc-0.48.5/lib" "/LIBPATH:/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/build/pyo3-ffi-d909cdc4c75b800d/out/lib" "/LIBPATH:/Users/peek/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/i686-pc-windows-msvc/lib" "/Users/peek/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/i686-pc-windows-msvc/lib/libcompiler_builtins-4aebb44556f09df4.rlib" "windows.0.48.5.lib" "python39.lib" "legacy_stdio_definitions.lib" "kernel32.lib" "advapi32.lib" "bcrypt.lib" "kernel32.lib" "ntdll.lib" "userenv.lib" "ws2_32.lib" "kernel32.lib" "ws2_32.lib" "kernel32.lib" "ntdll.lib" "msvcrt.lib" "/NXCOMPAT" "/LIBPATH:/Users/peek/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/i686-pc-windows-msvc/lib" "/OUT:/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/deps/ngc_rtc.dll" "/OPT:REF,ICF" "/DLL" "/IMPLIB:/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/deps/ngc_rtc.dll.lib" "/DEBUG:NONE" "-s"
  = note: lld-link: warning: ignoring unknown argument '-s'
          lld-link: error: could not open 'legacy_stdio_definitions.lib': No such file or directory
          lld-link: error: could not open 'kernel32.lib': No such file or directory
          lld-link: error: could not open 'advapi32.lib': No such file or directory
          lld-link: error: could not open 'bcrypt.lib': No such file or directory
          lld-link: error: could not open 'kernel32.lib': No such file or directory
          lld-link: error: could not open 'ntdll.lib': No such file or directory
          lld-link: error: could not open 'userenv.lib': No such file or directory
          lld-link: error: could not open 'ws2_32.lib': No such file or directory
          lld-link: error: could not open 'kernel32.lib': No such file or directory
          lld-link: error: could not open 'ws2_32.lib': No such file or directory
          lld-link: error: could not open 'kernel32.lib': No such file or directory
          lld-link: error: could not open 'ntdll.lib': No such file or directory
          lld-link: error: could not open 'msvcrt.lib': No such file or direct

On an m2 mac, but repro'd this same issue on an x86 linux box

poetry run maturin build --target i686-pc-windows-msvc --release --strip -i 3.9 --zig

Your maturin version (maturin --version)

maturin 1.4.0

Your Python version (python -V)

Python 3.9.18

Your pip version (pip -V)

pip 23.2.1

What bindings you're using

pyo3

Does cargo build work?

  • [x] Yes, it works (Doesn't work on m1 but works on lnx x86) maturin also works on every other target I've tried. Windows x86, mac arm, mac x86, linux arm, x86

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

  • [x] Yes

Steps to Reproduce

cat Cargo.toml

[package]
name = "ngc-runtime-commons"
version = "0.1.0"
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
pyo3 = { version = "0.20.2", features = ["extension-module"] }

[lib]
name = "ngc_rtc"
crate-type = ["dylib"]

[profile.release]
opt-level = 'z' # Optimize for size
lto = true
strip = true

Also tried (pyo3 = { version = "0.20.2", features = ["extension-module", "generate-import-lib"] }) but got same res.

RUST_LOG=maturin=debug  poetry run maturin build --target i686-pc-windows-msvc --release --strip -i 3.9 --zig
2024-02-22T16:26:00.457677Z DEBUG maturin::project_layout: Found pyproject.toml in working directory at "/Users/peek/dev/work/ngc-py-rust/pyproject.toml"
2024-02-22T16:26:00.457975Z DEBUG maturin::project_layout: Using cargo manifest path from working directory: "/Users/peek/dev/work/ngc-py-rust/Cargo.toml"
2024-02-22T16:26:00.458420Z DEBUG maturin::project_layout: Resolving cargo metadata from "/Users/peek/dev/work/ngc-py-rust/Cargo.toml"
2024-02-22T16:26:00.522539Z DEBUG maturin::project_layout: Project layout resolved project_root=/Users/peek/dev/work/ngc-py-rust python_dir=/Users/peek/dev/work/ngc-py-rust rust_module=/Users/peek/dev/work/ngc-py-rust/ngc_rtc python_module=/Users/peek/dev/work/ngc-py-rust/ngc_rtc extension_name=ngc_rtc module_name=ngc_rtc
🔗 Found pyo3 bindings
2024-02-22T16:26:00.554301Z DEBUG maturin::build_options: Found CPython 3.9 in bundled sysconfig
🐍 Found CPython 3.9
2024-02-22T16:26:00.554340Z DEBUG maturin::compile: Setting crate_type to cdylib for Rust >= 1.64.0
2024-02-22T16:26:00.715632Z DEBUG maturin::compile: Running env -u CARGO AR_i686_pc_windows_msvc="llvm-lib" BINDGEN_EXTRA_CLANG_ARGS_i686_pc_windows_msvc="-I/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared" CARGO_ENCODED_RUSTFLAGS="-C\u{1f}linker-flavor=lld-link\u{1f}-Lnative=/Users/peek/Library/Caches/cargo-xwin/xwin/crt/lib/x86\u{1f}-Lnative=/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/lib/um/x86\u{1f}-Lnative=/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/lib/ucrt/x86" CARGO_TARGET_I686_PC_WINDOWS_MSVC_LINKER="lld-link" CC_i686_pc_windows_msvc="clang-cl" CFLAGS_i686_pc_windows_msvc="--target=i686-pc-windows-msvc -Wno-unused-command-line-argument -fuse-ld=lld-link /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared  " CL_FLAGS="--target=i686-pc-windows-msvc -Wno-unused-command-line-argument -fuse-ld=lld-link /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared " CMAKE_GENERATOR="Ninja" CMAKE_SYSTEM_NAME="Windows" CMAKE_TOOLCHAIN_FILE_i686_pc_windows_msvc="/Users/peek/Library/Caches/cargo-xwin/cmake/i686-pc-windows-msvc-toolchain.cmake" CXXFLAGS_i686_pc_windows_msvc="--target=i686-pc-windows-msvc -Wno-unused-command-line-argument -fuse-ld=lld-link /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared  " CXX_i686_pc_windows_msvc="clang-cl" PATH="/Users/peek/dev/work/ngc-py-rust/.venv/bin:/Users/peek/.rd/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/Library/Frameworks/Python.framework/Versions/3.9/bin:/Users/peek/.rd/bin:/Users/peek/.local/bin:/Users/peek/.cargo/bin:/opt/homebrew/bin:/Users/peek/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Users/peek/Library/Caches/cargo-xwin:/opt/homebrew/opt/llvm/bin" PYO3_CONFIG_FILE="/Users/peek/dev/work/ngc-py-rust/target/maturin/pyo3-config-i686-pc-windows-msvc-3.9.txt" RCFLAGS="-I/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared" TARGET_AR="llvm-lib" TARGET_CC="clang-cl" TARGET_CXX="clang-cl" ZIG_COMMAND="/opt/homebrew/bin/zig" "cargo" "rustc" "--target" "i686-pc-windows-msvc" "--message-format" "json-render-diagnostics" "--manifest-path" "/Users/peek/dev/work/ngc-py-rust/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib" "--" "-C" "link-arg=-s"
   Compiling pyo3-build-config v0.20.2
   Compiling pyo3-ffi v0.20.2
   Compiling pyo3 v0.20.2
   Compiling ngc-runtime-commons v0.1.0 (/Users/peek/dev/work/ngc-py-rust)
error: linking with `lld-link` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/Users/peek/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/Users/peek/dev/work/ngc-py-rust/.venv/bin:/Users/peek/.rd/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/Library/Frameworks/Python.framework/Versions/3.9/bin:/Users/peek/.rd/bin:/Users/peek/.local/bin:/Users/peek/.cargo/bin:/opt/homebrew/bin:/Users/peek/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Users/peek/Library/Caches/cargo-xwin:/opt/homebrew/opt/llvm/bin" VSLANG="1033" "lld-link" "-flavor" "link" "/DEF:/var/folders/4j/ngr_tv3x4412d67vc01xmjsh0000gp/T/rustcUBYRBc/lib.def" "/NOLOGO" "/LARGEADDRESSAWARE" "/SAFESEH" "/var/folders/4j/ngr_tv3x4412d67vc01xmjsh0000gp/T/rustcUBYRBc/symbols.o" "/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/deps/ngc_rtc.ngc_rtc.f35e48bc522c81cd-cgu.0.rcgu.o" "/LIBPATH:/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/deps" "/LIBPATH:/Users/peek/dev/work/ngc-py-rust/target/release/deps" "/LIBPATH:/Users/peek/Library/Caches/cargo-xwin/xwin/crt/lib/x86" "/LIBPATH:/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/lib/um/x86" "/LIBPATH:/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/lib/ucrt/x86" "/LIBPATH:/Users/peek/.cargo/registry/src/index.crates.io-6f17d22bba15001f/windows_i686_msvc-0.48.5/lib" "/LIBPATH:/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/build/pyo3-ffi-d909cdc4c75b800d/out/lib" "/LIBPATH:/Users/peek/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/i686-pc-windows-msvc/lib" "/Users/peek/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/i686-pc-windows-msvc/lib/libcompiler_builtins-4aebb44556f09df4.rlib" "windows.0.48.5.lib" "python39.lib" "legacy_stdio_definitions.lib" "kernel32.lib" "advapi32.lib" "bcrypt.lib" "kernel32.lib" "ntdll.lib" "userenv.lib" "ws2_32.lib" "kernel32.lib" "ws2_32.lib" "kernel32.lib" "ntdll.lib" "msvcrt.lib" "/NXCOMPAT" "/LIBPATH:/Users/peek/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/i686-pc-windows-msvc/lib" "/OUT:/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/deps/ngc_rtc.dll" "/OPT:REF,ICF" "/DLL" "/IMPLIB:/Users/peek/dev/work/ngc-py-rust/target/i686-pc-windows-msvc/release/deps/ngc_rtc.dll.lib" "/DEBUG:NONE" "-s"
  = note: lld-link: warning: ignoring unknown argument '-s'
          lld-link: error: could not open 'legacy_stdio_definitions.lib': No such file or directory
          lld-link: error: could not open 'kernel32.lib': No such file or directory
          lld-link: error: could not open 'advapi32.lib': No such file or directory
          lld-link: error: could not open 'bcrypt.lib': No such file or directory
          lld-link: error: could not open 'kernel32.lib': No such file or directory
          lld-link: error: could not open 'ntdll.lib': No such file or directory
          lld-link: error: could not open 'userenv.lib': No such file or directory
          lld-link: error: could not open 'ws2_32.lib': No such file or directory
          lld-link: error: could not open 'kernel32.lib': No such file or directory
          lld-link: error: could not open 'ws2_32.lib': No such file or directory
          lld-link: error: could not open 'kernel32.lib': No such file or directory
          lld-link: error: could not open 'ntdll.lib': No such file or directory
          lld-link: error: could not open 'msvcrt.lib': No such file or directory


error: could not compile `ngc-runtime-commons` (lib) due to 1 previous error
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `env -u CARGO AR_i686_pc_windows_msvc="llvm-lib" BINDGEN_EXTRA_CLANG_ARGS_i686_pc_windows_msvc="-I/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared" CARGO_ENCODED_RUSTFLAGS="-C\u{1f}linker-flavor=lld-link\u{1f}-Lnative=/Users/peek/Library/Caches/cargo-xwin/xwin/crt/lib/x86\u{1f}-Lnative=/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/lib/um/x86\u{1f}-Lnative=/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/lib/ucrt/x86" CARGO_TARGET_I686_PC_WINDOWS_MSVC_LINKER="lld-link" CC_i686_pc_windows_msvc="clang-cl" CFLAGS_i686_pc_windows_msvc="--target=i686-pc-windows-msvc -Wno-unused-command-line-argument -fuse-ld=lld-link /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared  " CL_FLAGS="--target=i686-pc-windows-msvc -Wno-unused-command-line-argument -fuse-ld=lld-link /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared " CMAKE_GENERATOR="Ninja" CMAKE_SYSTEM_NAME="Windows" CMAKE_TOOLCHAIN_FILE_i686_pc_windows_msvc="/Users/peek/Library/Caches/cargo-xwin/cmake/i686-pc-windows-msvc-toolchain.cmake" CXXFLAGS_i686_pc_windows_msvc="--target=i686-pc-windows-msvc -Wno-unused-command-line-argument -fuse-ld=lld-link /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um /imsvc/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared  " CXX_i686_pc_windows_msvc="clang-cl" PATH="/Users/peek/dev/work/ngc-py-rust/.venv/bin:/Users/peek/.rd/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/Library/Frameworks/Python.framework/Versions/3.9/bin:/Users/peek/.rd/bin:/Users/peek/.local/bin:/Users/peek/.cargo/bin:/opt/homebrew/bin:/Users/peek/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Users/peek/Library/Caches/cargo-xwin:/opt/homebrew/opt/llvm/bin" PYO3_CONFIG_FILE="/Users/peek/dev/work/ngc-py-rust/target/maturin/pyo3-config-i686-pc-windows-msvc-3.9.txt" RCFLAGS="-I/Users/peek/Library/Caches/cargo-xwin/xwin/crt/include -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/ucrt -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/um -I/Users/peek/Library/Caches/cargo-xwin/xwin/sdk/include/shared" TARGET_AR="llvm-lib" TARGET_CC="clang-cl" TARGET_CXX="clang-cl" ZIG_COMMAND="/opt/homebrew/bin/zig" "cargo" "rustc" "--target" "i686-pc-windows-msvc" "--message-format" "json-render-diagnostics" "--manifest-path" "/Users/peek/dev/work/ngc-py-rust/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib" "--" "-C" "link-arg=-s"`

peeksake avatar Feb 22 '24 16:02 peeksake

Might be worth mentioning I've seen this issue with cargo xwin build --release --target i686-pc-windows-msvc and the fix was to add a flag --xwin-arch=x86 I tried adding this to the maturin run command but it just ended up appending it to the rustc flags which i'm sure is not what xwin-build does.

Going to dig a bit into the source code to see if I can add this.

peeksake avatar Feb 22 '24 16:02 peeksake

Okay, dug a little bit, looks like the default values for XWinOptions are x86_64 and AArch64, I was trying to set it with

if target.get_platform_arch().unwrap() == "i686" {
       build.xwin.xwin_arch = vec![xwin::Arch::x86];
}

But the xwin::Arch struct isn't importable as it collides with the conditional compilation name? (Not sure about this)

peeksake avatar Feb 22 '24 19:02 peeksake