text-embeddings-inference icon indicating copy to clipboard operation
text-embeddings-inference copied to clipboard
verified publisher icon huggingface

Compile error in multiple environments for CPU backend

Open nkh0472 opened this issue 3 months ago • 12 comments

System Info

TEI source code:

  • Latest main branch(0c1009bfc49b759fe75eed4fd377b4fbad534ad5);
  • Latest release v1.8.2;
  • Release v1.8.1

Tested platform:

  • Win: AMD 7950X+Windows 10 x64 Version 10.0.19045.6332;
  • WSL2: AMD 7950X+Debian 13 on wsl2 (Linux DESKTOP 5.15.167.4-microsoft-standard-WSL2 # 1 SMP Tue Nov 5 00:21:55 UTC 2024 x86_64 GNU/Linux) @ Windows 10 x64 Version 10.0.19045.6332;
  • Linux: Intel 6133*2+Ubuntu 20.04;

(GPUs is not mentioned due to build TEI on CPU)

Tested rustup envs:

Freshly installed rustup: default rustup profile: cargo 1.85.1 (d73d2caf9 2024-12-31)

  • Win: Freshly installed rustup & Freshly installed MSVC v143 -VS 2022 C++ build tools+Winodws 11 SDK (10.0.22621.0)+cmake
  • WSL: Freshly installed rustup & gcc (Debian 14.2.0-19) 14.2.0
  • Linux: Freshly installed rustup & gcc (GCC) 10.5.0

Information

  • [ ] Docker
  • [x] The CLI directly

Tasks

  • [x] An officially supported command
  • [ ] My own modifications

Reproduction

As docs' recommend, tested on 3 different envs listed above:

  1. curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  2. cargo install --path router -F mkl --verbose (added --verbose for logging)

Shows compile error about 25 undefined references / external symbol ('vsTanh', 'vsSub', 'vsSqrt', 'vsSin', 'vsMul', 'vsLn', 'vsFmin', 'vsExp', 'vsDiv', 'vsCos', 'vsAdd', 'vdTanh', 'vdSub', 'vdSqrt', 'vdSin', 'vdMul', 'vdLn', 'vdFmin', 'vdExp', 'vdDiv', 'vdCos', 'vdAdd', 'sgemm_', 'hgemm_', 'dgemm_')

Expected behavior

Expect finishing compile, but:

  • Compile v1.8.2/v1.8.1/main (similar error) on Win+MSVC+AMD CPU:
...
Running `C:\Users\nkh04\.rustup\toolchains\1.85.1-x86_64-pc-windows-msvc\bin\rustc.exe --crate-name text_embeddings_router --edition=2021 router\src\main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=115 --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C lto=fat -C codegen-units=1 --cfg "feature=\"candle\"" --cfg "feature=\"default\"" --cfg "feature=\"dynamic-linking\"" --cfg "feature=\"http\"" --cfg "feature=\"mkl\"" --check-cfg cfg(docsrs,test) --check-cfg "cfg(feature, values(\"accelerate\", \"candle\", \"candle-cuda\", \"candle-cuda-turing\", \"candle-cuda-volta\", \"default\", \"dynamic-linking\", \"google\", \"grpc\", \"http\", \"metal\", \"mkl\", \"ort\", \"python\", \"static-linking\"))" -C metadata=e1406d246b8c925f --out-dir F:\text-embeddings-inference-1.8.2\target\release\deps -C strip=symbols -L dependency=F:\text-embeddings-inference-1.8.2\target\release\deps --extern anyhow=F:\text-embeddings-inference-1.8.2\target\release\deps\libanyhow-5751be73768123a3.rlib --extern axum=F:\text-embeddings-inference-1.8.2\target\release\deps\libaxum-8bc59cf51b8d1ae2.rlib --extern axum_tracing_opentelemetry=F:\text-embeddings-inference-1.8.2\target\release\deps\libaxum_tracing_opentelemetry-6919ca207315f42e.rlib --extern base64=F:\text-embeddings-inference-1.8.2\target\release\deps\libbase64-20907aaabfa37a5c.rlib --extern clap=F:\text-embeddings-inference-1.8.2\target\release\deps\libclap-ded1b8a7f6da29a7.rlib --extern futures=F:\text-embeddings-inference-1.8.2\target\release\deps\libfutures-55e1ce906ca8ce43.rlib --extern hf_hub=F:\text-embeddings-inference-1.8.2\target\release\deps\libhf_hub-46162d037bf61d01.rlib --extern http=F:\text-embeddings-inference-1.8.2\target\release\deps\libhttp-721bb5a8d4ad5af4.rlib --extern init_tracing_opentelemetry=F:\text-embeddings-inference-1.8.2\target\release\deps\libinit_tracing_opentelemetry-1130e5d6b02b3c83.rlib --extern intel_mkl_src=F:\text-embeddings-inference-1.8.2\target\release\deps\libintel_mkl_src-7de47f7e38d141d5.rlib --extern metrics=F:\text-embeddings-inference-1.8.2\target\release\deps\libmetrics-f38f63f59a9e401d.rlib --extern metrics_exporter_prometheus=F:\text-embeddings-inference-1.8.2\target\release\deps\libmetrics_exporter_prometheus-3e83484daaaf9a40.rlib --extern mimalloc=F:\text-embeddings-inference-1.8.2\target\release\deps\libmimalloc-55786f97dafb497c.rlib --extern num_cpus=F:\text-embeddings-inference-1.8.2\target\release\deps\libnum_cpus-26f3f7fb7d16b825.rlib --extern opentelemetry=F:\text-embeddings-inference-1.8.2\target\release\deps\libopentelemetry-43ce590757d45ebb.rlib --extern opentelemetry_otlp=F:\text-embeddings-inference-1.8.2\target\release\deps\libopentelemetry_otlp-7adf99fb9a924955.rlib --extern opentelemetry_sdk=F:\text-embeddings-inference-1.8.2\target\release\deps\libopentelemetry_sdk-48d11cd15d38a406.rlib --extern reqwest=F:\text-embeddings-inference-1.8.2\target\release\deps\libreqwest-cdbb64c7917c22c9.rlib --extern serde=F:\text-embeddings-inference-1.8.2\target\release\deps\libserde-e13a1b310cb83bc5.rlib --extern serde_json=F:\text-embeddings-inference-1.8.2\target\release\deps\libserde_json-c2074a4721fb3f74.rlib --extern simsimd=F:\text-embeddings-inference-1.8.2\target\release\deps\libsimsimd-5bf7050b419eab84.rlib --extern text_embeddings_backend=F:\text-embeddings-inference-1.8.2\target\release\deps\libtext_embeddings_backend-47110aae34d263ff.rlib --extern text_embeddings_core=F:\text-embeddings-inference-1.8.2\target\release\deps\libtext_embeddings_core-fcfed64cb789ee6c.rlib --extern text_embeddings_router=F:\text-embeddings-inference-1.8.2\target\release\deps\libtext_embeddings_router-86d8337692d08222.rlib --extern thiserror=F:\text-embeddings-inference-1.8.2\target\release\deps\libthiserror-c6c427f0b26980bb.rlib --extern tokenizers=F:\text-embeddings-inference-1.8.2\target\release\deps\libtokenizers-c70c4878614e459c.rlib --extern tokio=F:\text-embeddings-inference-1.8.2\target\release\deps\libtokio-de8a54102807469d.rlib --extern tower_http=F:\text-embeddings-inference-1.8.2\target\release\deps\libtower_http-db8c27fe31b61d22.rlib --extern tracing=F:\text-embeddings-inference-1.8.2\target\release\deps\libtracing-9b14eddb7a0e050e.rlib --extern tracing_opentelemetry=F:\text-embeddings-inference-1.8.2\target\release\deps\libtracing_opentelemetry-58b0ef20e2cb901a.rlib --extern tracing_subscriber=F:\text-embeddings-inference-1.8.2\target\release\deps\libtracing_subscriber-36b789db4e13d6cf.rlib --extern utoipa=F:\text-embeddings-inference-1.8.2\target\release\deps\libutoipa-c4783a5f64f55a69.rlib --extern utoipa_swagger_ui=F:\text-embeddings-inference-1.8.2\target\release\deps\libutoipa_swagger_ui-495e29e9947ba5eb.rlib --extern veil=F:\text-embeddings-inference-1.8.2\target\release\deps\libveil-004a3642849f498c.rlib -C target-cpu=native -L native=C:\Users\nkh04\.cargo\registry\src\index.crates.io-1949cf8c6b5b557f\windows_x86_64_msvc-0.52.6\lib -L native=C:\Users\nkh04\.cargo\registry\src\index.crates.io-1949cf8c6b5b557f\windows_x86_64_msvc-0.53.0\lib -L native=F:\text-embeddings-inference-1.8.2\target\release\build\aws-lc-sys-780fb5839103b09c\out\build\artifacts -L native=F:\text-embeddings-inference-1.8.2\target\release\build\libmimalloc-sys-075948c0b2c1d0c4\out -L native=F:\text-embeddings-inference-1.8.2\target\release\build\simsimd-36c5bc20991f123d\out -L native=F:\text-embeddings-inference-1.8.2\target\release\build\esaxx-rs-f408cee4bdc7cd02\out -L native=F:\text-embeddings-inference-1.8.2\target\release\build\onig_sys-c4d07026b1e4ebc6\out`
> error: linking with `link.exe` failed: exit code: 1120
>   |
>   = note: "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.44.35207\\bin\\HostX64\\x64\\link.exe" "/NOLOGO" "X:\\Temp\\rustcHfzNk1\\symbols.o" "<1 object files omitted>" "X:\\Temp\\rustcHfzNk1/{libesaxx_rs-6e57b48aaf17cfd5.rlib,libonig_sys-ac9856d89e784b1f.rlib,libsimsimd-5bf7050b419eab84.rlib,libaws_lc_sys-e3a60ba08b12aba2.rlib,liblibmimalloc_sys-34fa5f039a2a27c0.rlib,liblibloading-11a73e2559dd28d8.rlib,libwindows_sys-ed084a5145e1664d.rlib,libwindows_registry-c858cc3d5258ebb1.rlib,libwindows_strings-747cadb66d822a53.rlib,libwindows_result-4d9f5116e07ea188.rlib,libgetrandom-8148a8b002c96228.rlib,libstd-dd4d958a589b8369.rlib}" "C:\\Users\\nkh04\\.rustup\\toolchains\\1.85.1-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib/{libcompiler_builtins-34b1b72af8949504.rlib}" "kernel32.lib" "C:\\Users\\nkh04\\.cargo\\registry\\src\\index.crates.io-1949cf8c6b5b557f\\windows_x86_64_msvc-0.52.6\\lib\\windows.0.52.0.lib" "bcrypt.lib" "advapi32.lib" "psapi.lib" "shell32.lib" "user32.lib" "advapi32.lib" "bcrypt.lib" "legacy_stdio_definitions.lib" "C:\\Users\\nkh04\\.cargo\\registry\\src\\index.crates.io-1949cf8c6b5b557f\\windows_x86_64_msvc-0.52.6\\lib\\windows.0.52.0.lib" "C:\\Users\\nkh04\\.cargo\\registry\\src\\index.crates.io-1949cf8c6b5b557f\\windows_x86_64_msvc-0.52.6\\lib\\windows.0.52.0.lib" "C:\\Users\\nkh04\\.cargo\\registry\\src\\index.crates.io-1949cf8c6b5b557f\\windows_x86_64_msvc-0.53.0\\lib\\windows.0.53.0.lib" "kernel32.lib" "kernel32.lib" "advapi32.lib" "ntdll.lib" "userenv.lib" "ws2_32.lib" "dbghelp.lib" "/defaultlib:msvcrt" "/NXCOMPAT" "/LIBPATH:C:\\Users\\nkh04\\.cargo\\registry\\src\\index.crates.io-1949cf8c6b5b557f\\windows_x86_64_msvc-0.52.6\\lib" "/LIBPATH:C:\\Users\\nkh04\\.cargo\\registry\\src\\index.crates.io-1949cf8c6b5b557f\\windows_x86_64_msvc-0.53.0\\lib" "/LIBPATH:F:\\text-embeddings-inference-1.8.2\\target\\release\\build\\aws-lc-sys-780fb5839103b09c\\out\\build\\artifacts" "/LIBPATH:F:\\text-embeddings-inference-1.8.2\\target\\release\\build\\libmimalloc-sys-075948c0b2c1d0c4\\out" "/LIBPATH:F:\\text-embeddings-inference-1.8.2\\target\\release\\build\\simsimd-36c5bc20991f123d\\out" "/LIBPATH:F:\\text-embeddings-inference-1.8.2\\target\\release\\build\\esaxx-rs-f408cee4bdc7cd02\\out" "/LIBPATH:F:\\text-embeddings-inference-1.8.2\\target\\release\\build\\onig_sys-c4d07026b1e4ebc6\\out" "/OUT:F:\\text-embeddings-inference-1.8.2\\target\\release\\deps\\text_embeddings_router.exe" "/OPT:REF,ICF" "/DEBUG" "/PDBALTPATH:%_PDB%" "/NATVIS:C:\\Users\\nkh04\\.rustup\\toolchains\\1.85.1-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\nkh04\\.rustup\\toolchains\\1.85.1-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\nkh04\\.rustup\\toolchains\\1.85.1-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\nkh04\\.rustup\\toolchains\\1.85.1-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis" "/NATVIS:X:\\Temp\\rustcHfzNk1\\text_embeddings_router-0.natvis" "/NATVIS:X:\\Temp\\rustcHfzNk1\\text_embeddings_router-1.natvis"
>   = note: some arguments are omitted. use `--verbose` to show all linker arguments
>   = note:   正在创建库 F:\text-embeddings-inference-1.8.2\target\release\deps\text_embeddings_router.lib 和对象 F:\text-embeddings-inference-1.8.2\target\release\deps\text_embeddings_router.exp␍
>           text_embeddings_router.text_embeddings_router.7c55587b9b535090-cgu.0.rcgu.o : error LNK2019: unresolved external symbol vsMul，referenced in function _ZN11candle_core6tensor6Tensor3sqr17h629c89a1168adab6E␍
>           text_embeddings_router.text_embeddings_router.7c55587b9b535090-cgu.0.rcgu.o : error LNK2019: unresolved external symbol vdMul，referenced in function _ZN11candle_core6tensor6Tensor3sqr17h629c89a1168adab6E␍
...
>           F:\text-embeddings-inference-1.8.2\target\release\deps\text_embeddings_router.exe : fatal error LNK1120: 25 unresolved externals␍
> 
> 
> error: could not compile `text-embeddings-router` (bin "text-embeddings-router") due to 1 previous error
> 
> Caused by:
>   process didn't exit successfully: `C:\Users\nkh04\.rustup\toolchains\1.85.1-x86_64-pc-windows-msvc\bin\rustc.exe --crate-name text_embeddings_router --edition=2021 router\src\main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=115 --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C lto=fat -C codegen-units=1 --cfg "feature=\"candle\"" --cfg "feature=\"default\"" --cfg "feature=\"dynamic-linking\"" --cfg "feature=\"http\"" --cfg "feature=\"mkl\"" --check-cfg cfg(docsrs,test) --check-cfg "cfg(feature, values(\"accelerate\", \"candle\", \"candle-cuda\", \"candle-cuda-turing\", \"candle-cuda-volta\", \"default\", \"dynamic-linking\", \"google\", \"grpc\", \"http\", \"metal\", \"mkl\", \"ort\", \"python\", \"static-linking\"))" -C metadata=e1406d246b8c925f --out-dir F:\text-embeddings-inference-1.8.2\target\release\deps -C strip=symbols -L dependency=F:\text-embeddings-inference-1.8.2\target\release\deps --extern anyhow=F:\text-embeddings-inference-1.8.2\target\release\deps\libanyhow-5751be73768123a3.rlib --extern axum=F:\text-embeddings-inference-1.8.2\target\release\deps\libaxum-8bc59cf51b8d1ae2.rlib --extern axum_tracing_opentelemetry=F:\text-embeddings-inference-1.8.2\target\release\deps\libaxum_tracing_opentelemetry-6919ca207315f42e.rlib --extern base64=F:\text-embeddings-inference-1.8.2\target\release\deps\libbase64-20907aaabfa37a5c.rlib --extern clap=F:\text-embeddings-inference-1.8.2\target\release\deps\libclap-ded1b8a7f6da29a7.rlib --extern futures=F:\text-embeddings-inference-1.8.2\target\release\deps\libfutures-55e1ce906ca8ce43.rlib --extern hf_hub=F:\text-embeddings-inference-1.8.2\target\release\deps\libhf_hub-46162d037bf61d01.rlib --extern http=F:\text-embeddings-inference-1.8.2\target\release\deps\libhttp-721bb5a8d4ad5af4.rlib --extern init_tracing_opentelemetry=F:\text-embeddings-inference-1.8.2\target\release\deps\libinit_tracing_opentelemetry-1130e5d6b02b3c83.rlib --extern intel_mkl_src=F:\text-embeddings-inference-1.8.2\target\release\deps\libintel_mkl_src-7de47f7e38d141d5.rlib --extern metrics=F:\text-embeddings-inference-1.8.2\target\release\deps\libmetrics-f38f63f59a9e401d.rlib --extern metrics_exporter_prometheus=F:\text-embeddings-inference-1.8.2\target\release\deps\libmetrics_exporter_prometheus-3e83484daaaf9a40.rlib --extern mimalloc=F:\text-embeddings-inference-1.8.2\target\release\deps\libmimalloc-55786f97dafb497c.rlib --extern num_cpus=F:\text-embeddings-inference-1.8.2\target\release\deps\libnum_cpus-26f3f7fb7d16b825.rlib --extern opentelemetry=F:\text-embeddings-inference-1.8.2\target\release\deps\libopentelemetry-43ce590757d45ebb.rlib --extern opentelemetry_otlp=F:\text-embeddings-inference-1.8.2\target\release\deps\libopentelemetry_otlp-7adf99fb9a924955.rlib --extern opentelemetry_sdk=F:\text-embeddings-inference-1.8.2\target\release\deps\libopentelemetry_sdk-48d11cd15d38a406.rlib --extern reqwest=F:\text-embeddings-inference-1.8.2\target\release\deps\libreqwest-cdbb64c7917c22c9.rlib --extern serde=F:\text-embeddings-inference-1.8.2\target\release\deps\libserde-e13a1b310cb83bc5.rlib --extern serde_json=F:\text-embeddings-inference-1.8.2\target\release\deps\libserde_json-c2074a4721fb3f74.rlib --extern simsimd=F:\text-embeddings-inference-1.8.2\target\release\deps\libsimsimd-5bf7050b419eab84.rlib --extern text_embeddings_backend=F:\text-embeddings-inference-1.8.2\target\release\deps\libtext_embeddings_backend-47110aae34d263ff.rlib --extern text_embeddings_core=F:\text-embeddings-inference-1.8.2\target\release\deps\libtext_embeddings_core-fcfed64cb789ee6c.rlib --extern text_embeddings_router=F:\text-embeddings-inference-1.8.2\target\release\deps\libtext_embeddings_router-86d8337692d08222.rlib --extern thiserror=F:\text-embeddings-inference-1.8.2\target\release\deps\libthiserror-c6c427f0b26980bb.rlib --extern tokenizers=F:\text-embeddings-inference-1.8.2\target\release\deps\libtokenizers-c70c4878614e459c.rlib --extern tokio=F:\text-embeddings-inference-1.8.2\target\release\deps\libtokio-de8a54102807469d.rlib --extern tower_http=F:\text-embeddings-inference-1.8.2\target\release\deps\libtower_http-db8c27fe31b61d22.rlib --extern tracing=F:\text-embeddings-inference-1.8.2\target\release\deps\libtracing-9b14eddb7a0e050e.rlib --extern tracing_opentelemetry=F:\text-embeddings-inference-1.8.2\target\release\deps\libtracing_opentelemetry-58b0ef20e2cb901a.rlib --extern tracing_subscriber=F:\text-embeddings-inference-1.8.2\target\release\deps\libtracing_subscriber-36b789db4e13d6cf.rlib --extern utoipa=F:\text-embeddings-inference-1.8.2\target\release\deps\libutoipa-c4783a5f64f55a69.rlib --extern utoipa_swagger_ui=F:\text-embeddings-inference-1.8.2\target\release\deps\libutoipa_swagger_ui-495e29e9947ba5eb.rlib --extern veil=F:\text-embeddings-inference-1.8.2\target\release\deps\libveil-004a3642849f498c.rlib -C target-cpu=native -L native=C:\Users\nkh04\.cargo\registry\src\index.crates.io-1949cf8c6b5b557f\windows_x86_64_msvc-0.52.6\lib -L native=C:\Users\nkh04\.cargo\registry\src\index.crates.io-1949cf8c6b5b557f\windows_x86_64_msvc-0.53.0\lib -L native=F:\text-embeddings-inference-1.8.2\target\release\build\aws-lc-sys-780fb5839103b09c\out\build\artifacts -L native=F:\text-embeddings-inference-1.8.2\target\release\build\libmimalloc-sys-075948c0b2c1d0c4\out -L native=F:\text-embeddings-inference-1.8.2\target\release\build\simsimd-36c5bc20991f123d\out -L native=F:\text-embeddings-inference-1.8.2\target\release\build\esaxx-rs-f408cee4bdc7cd02\out -L native=F:\text-embeddings-inference-1.8.2\target\release\build\onig_sys-c4d07026b1e4ebc6\out` (exit code: 1)
> error: failed to compile `text-embeddings-router v1.8.2 (F:\text-embeddings-inference-1.8.2\router)`, intermediate artifacts can be found at `F:\text-embeddings-inference-1.8.2\target`.
  • Compile v1.8.2 on WSL2+gcc 14+AMD CPU:
...
Running `/home/n/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/bin/rustc --crate-name text_embeddings_router --edition=2021 router/src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=116 --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C lto=fat -C codegen-units=1 --cfg 'feature="candle"' --cfg 'feature="default"' --cfg 'feature="dynamic-linking"' --cfg 'feature="http"' --cfg 'feature="mkl"' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values("accelerate", "candle", "candle-cuda", "candle-cuda-turing", "candle-cuda-volta", "default", "dynamic-linking", "google", "grpc", "http", "metal", "mkl", "ort", "python", "static-linking"))' -C metadata=1841d6e377b487fe -C extra-filename=-7a258fa1baba3ea3 --out-dir /mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps -C strip=symbols -L dependency=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps --extern anyhow=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libanyhow-4660ba28df6f7c9a.rlib --extern axum=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libaxum-46c83e23a09d05dd.rlib --extern axum_tracing_opentelemetry=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libaxum_tracing_opentelemetry-05d4324d72871a56.rlib --extern base64=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libbase64-45c66f817e4078b8.rlib --extern clap=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libclap-a8bdfbe4b8b01368.rlib --extern futures=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libfutures-cadba558f29334f5.rlib --extern hf_hub=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libhf_hub-2270c21865473c70.rlib --extern http=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libhttp-1e1220ba9590e870.rlib --extern init_tracing_opentelemetry=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libinit_tracing_opentelemetry-18a3e7b4a17e97be.rlib --extern intel_mkl_src=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libintel_mkl_src-6aa9ceabd76140dd.rlib --extern libc=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/liblibc-0a7b01598885b7e1.rlib --extern metrics=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libmetrics-5231a4984ec5dbc6.rlib --extern metrics_exporter_prometheus=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libmetrics_exporter_prometheus-f76c6927764b53b0.rlib --extern num_cpus=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libnum_cpus-860516a882c060a0.rlib --extern opentelemetry=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libopentelemetry-2bfa186370b15a82.rlib --extern opentelemetry_otlp=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libopentelemetry_otlp-edfbf91085c60c40.rlib --extern opentelemetry_sdk=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libopentelemetry_sdk-f4b7c5d2b6c27253.rlib --extern reqwest=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libreqwest-ef80a1987228e3e3.rlib --extern serde=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libserde-142cc40a82007987.rlib --extern serde_json=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libserde_json-848b11166a855cef.rlib --extern simsimd=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libsimsimd-e642e88b7c150607.rlib --extern text_embeddings_backend=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtext_embeddings_backend-c45d1a1b3cfdc388.rlib --extern text_embeddings_core=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtext_embeddings_core-1dacd137eff0ce0b.rlib --extern text_embeddings_router=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtext_embeddings_router-ca31db07b38650a4.rlib --extern thiserror=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libthiserror-4d538c63670ab48b.rlib --extern tokenizers=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtokenizers-0d9b26b47c3c0968.rlib --extern tokio=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtokio-c09adedaec5226f9.rlib --extern tower_http=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtower_http-4963fb0c66f4eb86.rlib --extern tracing=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtracing-5e56e748ade71738.rlib --extern tracing_opentelemetry=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtracing_opentelemetry-60c770ffce7b7e29.rlib --extern tracing_subscriber=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libtracing_subscriber-af03fa1cb28edf7b.rlib --extern utoipa=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libutoipa-a67be0f1e8807efe.rlib --extern utoipa_swagger_ui=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libutoipa_swagger_ui-dbc4bd29561afeec.rlib --extern veil=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/libveil-59cfab4f390256bb.rlib -C target-cpu=native -L native=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/build/aws-lc-sys-36da19d0f9c47d3b/out -L native=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/build/simsimd-08e34d61368bf0c3/out -L native=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/build/esaxx-rs-b7b08fd242b53a8e/out -L native=/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/build/onig_sys-e7ba94824b04f014/out`
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/n/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/n/.cargo/bin:/home/n/miniforge3/bin:/home/n/miniforge3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program Files (x86)/Common Files/MVS/Runtime/Win32_i86:/mnt/c/Program Files (x86)/Common Files/MVS/Runtime/Win64_x64:/mnt/c/Program Files (x86)/Thermo/Foundation/x64:/mnt/c/Windows/system32:/mnt/c/Windows:/mnt/c/Windows/System32/Wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/d/Libraries:/mnt/d/Software/nTrun/Goto:/mnt/c/Program Files/dotnet:/mnt/d/Libraries/Java/jdk8_x64/jre/bin:/mnt/d/Software/PDFtk Server/bin:/mnt/d/Office/MATLAB2023b/runtime/win64:/mnt/d/Office/MATLAB2023b/bin:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/Program Files (x86)/Thermo/Foundation:/mnt/d/msys64/ucrt64/bin:/mnt/d/msys64/ucrt64/x86_64-w64-mingw32/bin:/mnt/d/msys64/usr/bin:/mnt/c/Program Files/Docker/Docker/resources/bin:/mnt/c/Program Files/Wolfram Research/WolframScript:/mnt/d/Software/Bitvise SSH Client:/mnt/c/Users/nkh04/.cargo/bin:/mnt/d/Office/texlive/2025/bin/windows:/mnt/d/Libraries/miniforge3:/mnt/d/Libraries/miniforge3/Library/mingw-w64/bin:/mnt/d/Libraries/miniforge3/Library/usr/bin:/mnt/d/Libraries/miniforge3/Library/bin:/mnt/d/Libraries/miniforge3/Scripts:/mnt/c/Users/nkh04/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/nkh04/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/nkh04/.lmstudio/bin:/mnt/c/Users/nkh04/AppData/Local/Programs/Ollama:/mnt/d/Software/OpenMS-3.3.0/bin:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/Comet:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/LuciPHOr2:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/MaRaCluster:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/MSGFPlus:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/Percolator:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/pwiz-bin:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/SpectraST:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/ThermoRawFileParser:/mnt/d/Software/OpenMS-3.3.0/share/OpenMS/THIRDPARTY/XTandem:/mnt/c/Program Files/cpolar" VSLANG="1033" "cc" "-m64" "/tmp/rustcYmrgCR/symbols.o" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/tmp/rustcYmrgCR/{libesaxx_rs-e75533045cd99d2f.rlib,libonig_sys-e47cc43c9997aa8d.rlib,libsimsimd-e642e88b7c150607.rlib,libaws_lc_sys-298ab5103f7b8529.rlib}" "/home/n/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libcompiler_builtins-6497188c539eb018.rlib}" "-Wl,-Bdynamic" "-lstdc++" "-ldl" "-lssl" "-lcrypto" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/build/aws-lc-sys-36da19d0f9c47d3b/out" "-L" "/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/build/simsimd-08e34d61368bf0c3/out" "-L" "/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/build/esaxx-rs-b7b08fd242b53a8e/out" "-L" "/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/build/onig_sys-e7ba94824b04f014/out" "-L" "/home/n/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/text_embeddings_router-7a258fa1baba3ea3" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: /usr/bin/ld: /mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/text_embeddings_router-7a258fa1baba3ea3.text_embeddings_router.4a30262377585bc-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::sqr':
          text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x7df): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x838): undefined reference to `vdMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x2573): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x2613): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x2a03): undefined reference to `vdMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x2ae4): undefined reference to `vdMul'
          /usr/bin/ld: /mnt/f/text-embeddings-inference-1.8.2-linux/target/release/deps/text_embeddings_router-7a258fa1baba3ea3.text_embeddings_router.4a30262377585bc-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::sin':
          text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17hbb791af634aefe6aE+0x7f3): undefined reference to `vsSin'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17hbb791af634aefe6aE+0x849): undefined reference to `vdSin'
...
          collect2: error: ld returned 1 exit status

  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib )

error: could not compile `text-embeddings-router` (bin "text-embeddings-router") due to 1 previous error

Caused by:
error: failed to compile `text-embeddings-router v1.8.2 (/mnt/f/text-embeddings-inference-1.8.2-linux/router)`, intermediate artifacts can be found at `/mnt/f/text-embeddings-inference-1.8.2-linux/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
  • Compile v1.8.2 on Linux+gcc 10.5+Intel CPU:
...
Compiling text-embeddings-core v1.8.2 (/home/usr1/text-embeddings-inference-1.8.2/core)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/home/usr1/.cargo/bin:/home/usr1/.local/intelpython3/bin:/home/usr1/.local/intelpython3/condabin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustctR2j3y/symbols.o" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/tmp/rustctR2j3y/{libesaxx_rs-e75533045cd99d2f.rlib,libonig_sys-e47cc43c9997aa8d.rlib,libsimsimd-e642e88b7c150607.rlib,libaws_lc_sys-298ab5103f7b8529.rlib}" "/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libcompiler_builtins-6497188c539eb018.rlib}" "-Wl,-Bdynamic" "-lstdc++" "-ldl" "-lssl" "-lcrypto" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/usr1/.local/lib64" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/aws-lc-sys-36da19d0f9c47d3b/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/simsimd-08e34d61368bf0c3/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/esaxx-rs-b7b08fd242b53a8e/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/onig_sys-e7ba94824b04f014/out" "-L" "/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-7a258fa1baba3ea3" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-7a258fa1baba3ea3.text_embeddings_router.4a30262377585bc-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::sqr':
          text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x7c3): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x819): undefined reference to `vdMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x2410): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x24ac): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x28a0): undefined reference to `vdMul'
          /usr/bin/ld: text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17h27ec3979f66e6f5eE+0x2980): undefined reference to `vdMul'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-7a258fa1baba3ea3.text_embeddings_router.4a30262377585bc-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::sin':
          text_embeddings_router.4a30262377585bc-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17hbb791af634aefe6aE+0x7ea): undefined reference to `vsSin'
...
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `text-embeddings-router` (bin "text-embeddings-router") due to 1 previous error
error: failed to compile `text-embeddings-router v1.8.2 (/home/usr1/text-embeddings-inference-1.8.2/router)`, intermediate artifacts can be found at `/home/usr1/text-embeddings-inference-1.8.2/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

compile logs.txt

nkh0472 avatar Sep 25 '25 11:09 nkh0472

This may relate to MKL? #715

nkh0472 avatar Sep 25 '25 12:09 nkh0472

Hey @nkh0472 thanks for reporting and for the detailed logs!

AFAIK that might be related to default linking for Intel MKL, as when enabling the --feature mkl (in the aforementioned command cargo install --path router -F mkl) means it comes with mkl and the default features which are candle, http, and dynamic-linking. Given that mkl requires any of static-linking or dynamic-linking to link the Intel MKL libraries, the error suggests that dynamic-linking is not working as expected hence failing.

For that I'd suggest to either provide the LD_LIBRARY_PATH=/opt/intel/oneapi/mkl/latest/lib/intel64 or to whatever location your Intel MKL installation is at, or rather try with --features candle,mkl,static-linking,http --no-default-features instead of dyanmic-linking in case that's problematic.

To install Intel MKL oneAPI you can either:

  • Follow the instructions at https://www.intel.com/content/www/us/en/docs/onemkl/get-started-guide/2025-0/overview.html
  • Or the instructions at https://github.com/rust-math/intel-mkl-src (which is the underlying crate that powers the Intel MKL optimizations within candle)

Finally, as per the AMD CPU instances, I'd suggest not using Intel MKL as it won't work out of the box unless you follow the steps at https://danieldk.eu/Intel-MKL-on-AMD-Zen (by @danieldk), but not sure that Intel MKL optimizations are worth the effort in this case, as we'd recommend always to use ONNX when applicable on CPU i.e., with --features ort,http --no-default-features instead to use the ONNX Runtime instead of Candle.

In any case, please do let me know if any of those solutions work for you 🤗

alvarobartt avatar Sep 25 '25 12:09 alvarobartt

Finally, as per the AMD CPU instances, I'd suggest not using Intel MKL as it won't work out of the box unless you follow the steps at https://danieldk.eu/Intel-MKL-on-AMD-Zen (by @danieldk), but not sure that Intel MKL optimizations are worth the effort in this case, as we'd recommend always to use ONNX when applicable on CPU i.e., with instead to use the ONNX Runtime instead of Candle.--features ort,http --no-default-features

In github README.md page, says recommend use ONNX on CPU; but official docs just recommend use MKL on x86 CPU. The doc should be updated?

Compile v1.8.2 (ONNX backend) succeed on Win+MSVC+AMD CPU: cargo install --path router -F ort

nkh0472 avatar Sep 25 '25 12:09 nkh0472

Yes @nkh0472 that's a great point, apologies I missed that! I'll create a PR to update the documentation to make sure that we recommend ONNX as the default option on CPU as proved to be more performant! 🤗

Also did you manage to try any of the aforementioned options for solving your issue with Intel MKL?

Lastly, I'd like to flag that even if ONNX is the recommended approach for running inference on CPU with Text Embeddings Inference (TEI), I like to flag that some models might not come with ONNX weights within their Hugging Face Hub repository e.g. https://huggingface.co/google/embeddinggemma-300m, so you might need to convert those manually with either https://gist.github.com/tomaarsen/4b00b0e3be8884efa64cfab9230b161f or with https://huggingface.co/spaces/onnx-community/convert-to-onnx, or just rely on ONNX exports for such models as e.g. https://huggingface.co/onnx-community/embeddinggemma-300m-ONNX.

This being said, ONNX is the recommended approach but given that some models might lack the ONNX weights within the repository, the broader solution to support all of those is relying on candle with or without Intel MKL support, so the recommended would be cargo install --path router --feature ort or rather cargo install --path router --features mkl,ort instead if Intel MKL support is preferred than plain candle.

alvarobartt avatar Sep 25 '25 13:09 alvarobartt

Compile v1.8.2 (MKL backend) faild on Linux+gcc 10.5+Intel CPU+oneMKL 2025.2:

sh ./intel-onemkl-2025.2.0.629_offline.sh
# installed to ~/.local/oneapi/

export PKG_CONFIG_PATH=/home/usr1/.local/lib64/pkgconfig:/home/usr1/.local/oneapi/mkl/latest/lib/pkgconfig
export MKLROOT=/home/usr1/.local/oneapi/mkl
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/usr1/.local/oneapi/mkl/latest/lib
cargo install --path router -F mkl

Same errors like above linux envs shown:

   Compiling text-embeddings-router v1.8.2 (/home/usr1/text-embeddings-inference-1.8.2/router)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/home/usr1/.cargo/bin:/home/usr1/.local/intelpython3/envs/vllm/bin:/home/usr1/.local/intelpython3/condabin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustchPlQY0/symbols.o" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/tmp/rustchPlQY0/{libesaxx_rs-e75533045cd99d2f.rlib,libonig_sys-e47cc43c9997aa8d.rlib,libsimsimd-e642e88b7c150607.rlib,libaws_lc_sys-298ab5103f7b8529.rlib}" "/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libcompiler_builtins-6497188c539eb018.rlib}" "-Wl,-Bdynamic" "-lstdc++" "-ldl" "-lssl" "-lcrypto" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/usr1/.local/lib64" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/aws-lc-sys-36da19d0f9c47d3b/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/simsimd-08e34d61368bf0c3/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/esaxx-rs-b7b08fd242b53a8e/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/onig_sys-e7ba94824b04f014/out" "-L" "/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::sqr':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17he51a2f34348fb0f2E+0x7c3): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17he51a2f34348fb0f2E+0x819): undefined reference to `vdMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17he51a2f34348fb0f2E+0x2410): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17he51a2f34348fb0f2E+0x24ac): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17he51a2f34348fb0f2E+0x28a0): undefined reference to `vdMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sqr17he51a2f34348fb0f2E+0x2980): undefined reference to `vdMul'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::sin':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17h5ba11e524cdc27deE+0x7ea): undefined reference to `vsSin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17h5ba11e524cdc27deE+0x840): undefined reference to `vdSin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17h5ba11e524cdc27deE+0x26e2): undefined reference to `vsSin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17h5ba11e524cdc27deE+0x2738): undefined reference to `vsSin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17h5ba11e524cdc27deE+0x2b57): undefined reference to `vdSin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3sin17h5ba11e524cdc27deE+0x2bb2): undefined reference to `vdSin'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::tanh':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4tanh17h12c5bf1da183b712E+0x8a2): undefined reference to `vsTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4tanh17h12c5bf1da183b712E+0x902): undefined reference to `vdTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4tanh17h12c5bf1da183b712E+0x204d): undefined reference to `vsTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4tanh17h12c5bf1da183b712E+0x20ed): undefined reference to `vsTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4tanh17h12c5bf1da183b712E+0x24dd): undefined reference to `vdTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4tanh17h12c5bf1da183b712E+0x26f0): undefined reference to `vdTanh'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `<candle_core::cpu_backend::CpuStorage as candle_core::backend::BackendStorage>::matmul':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$6matmul17h6629f3e3bdd111efE+0x179b): undefined reference to `hgemm_'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$6matmul17h6629f3e3bdd111efE+0x1a28): undefined reference to `sgemm_'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$6matmul17h6629f3e3bdd111efE+0x1ca0): undefined reference to `dgemm_'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `<candle_core::cpu_backend::CpuStorage as candle_core::backend::BackendStorage>::binary_impl':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h970addeb6dd6ee48E+0x9f03): undefined reference to `vsSub'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h970addeb6dd6ee48E+0xaba8): undefined reference to `vsSub'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h970addeb6dd6ee48E+0xad9e): undefined reference to `vsSub'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h970addeb6dd6ee48E+0xc727): undefined reference to `vdSub'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h970addeb6dd6ee48E+0xd493): undefined reference to `vdSub'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h970addeb6dd6ee48E+0xd681): undefined reference to `vdSub'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `<candle_core::cpu_backend::CpuStorage as candle_core::backend::BackendStorage>::binary_impl':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h389758f807099b21E+0x9ee3): undefined reference to `vsAdd'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h389758f807099b21E+0xab68): undefined reference to `vsAdd'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h389758f807099b21E+0xad5e): undefined reference to `vsAdd'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h389758f807099b21E+0xc6a9): undefined reference to `vdAdd'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h389758f807099b21E+0xd40b): undefined reference to `vdAdd'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h389758f807099b21E+0xd5f9): undefined reference to `vdAdd'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `<candle_core::cpu_backend::CpuStorage as candle_core::backend::BackendStorage>::binary_impl':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h4311665d27674b83E+0xeac): undefined reference to `vsDiv'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h4311665d27674b83E+0x10a4): undefined reference to `vsDiv'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h4311665d27674b83E+0x13e7): undefined reference to `vsDiv'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h4311665d27674b83E+0x1ed4): undefined reference to `vdDiv'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h4311665d27674b83E+0x20c0): undefined reference to `vdDiv'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h4311665d27674b83E+0x23ff): undefined reference to `vdDiv'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `<candle_core::cpu_backend::CpuStorage as candle_core::backend::BackendStorage>::binary_impl':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h68e42c1adcbbf9c1E+0x9f34): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h68e42c1adcbbf9c1E+0xac00): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h68e42c1adcbbf9c1E+0xadee): undefined reference to `vsMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h68e42c1adcbbf9c1E+0xc6f7): undefined reference to `vdMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h68e42c1adcbbf9c1E+0xd42e): undefined reference to `vdMul'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17h68e42c1adcbbf9c1E+0xd614): undefined reference to `vdMul'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::log':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3log17he5d956cfed7f71d9E+0x7ea): undefined reference to `vsLn'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3log17he5d956cfed7f71d9E+0x840): undefined reference to `vdLn'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3log17he5d956cfed7f71d9E+0x26e2): undefined reference to `vsLn'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3log17he5d956cfed7f71d9E+0x2738): undefined reference to `vsLn'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3log17he5d956cfed7f71d9E+0x2b57): undefined reference to `vdLn'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor3log17he5d956cfed7f71d9E+0x2bb2): undefined reference to `vdLn'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::sqrt':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4sqrt17h7725717daaebfefcE+0x7c0): undefined reference to `vsSqrt'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4sqrt17h7725717daaebfefcE+0x813): undefined reference to `vdSqrt'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4sqrt17h7725717daaebfefcE+0x240c): undefined reference to `vsSqrt'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4sqrt17h7725717daaebfefcE+0x24a5): undefined reference to `vsSqrt'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4sqrt17h7725717daaebfefcE+0x288c): undefined reference to `vdSqrt'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4sqrt17h7725717daaebfefcE+0x2969): undefined reference to `vdSqrt'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::gelu':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0xa40): undefined reference to `vsTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0xa59): undefined reference to `vdTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0xe93): undefined reference to `vsTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0xefa): undefined reference to `vsTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0x10e5): undefined reference to `vdTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0x1145): undefined reference to `vdTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0x2a4d): undefined reference to `vsTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0x2c86): undefined reference to `vsTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0x31b8): undefined reference to `vdTanh'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4gelu17h67b2dde5a530be2fE+0x33f6): undefined reference to `vdTanh'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `candle_core::tensor::Tensor::silu':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x937): undefined reference to `vsExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x94a): undefined reference to `vdExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x1785): undefined reference to `vsExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x17ca): undefined reference to `vsExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x192a): undefined reference to `vdExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x1968): undefined reference to `vdExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x33fb): undefined reference to `vsExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x35a7): undefined reference to `vsExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x3b98): undefined reference to `vdExp'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN11candle_core6tensor6Tensor4silu17hd218ca4a05e6d87bE+0x3d47): undefined reference to `vdExp'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `text_embeddings_backend_candle::layers::rotary::get_cos_sin':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN30text_embeddings_backend_candle6layers6rotary11get_cos_sin17h5874782181534d74E+0xb61): undefined reference to `vsCos'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN30text_embeddings_backend_candle6layers6rotary11get_cos_sin17h5874782181534d74E+0xbb0): undefined reference to `vdCos'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN30text_embeddings_backend_candle6layers6rotary11get_cos_sin17h5874782181534d74E+0x2aa1): undefined reference to `vsCos'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN30text_embeddings_backend_candle6layers6rotary11get_cos_sin17h5874782181534d74E+0x2b05): undefined reference to `vsCos'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN30text_embeddings_backend_candle6layers6rotary11get_cos_sin17h5874782181534d74E+0x2ef3): undefined reference to `vdCos'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN30text_embeddings_backend_candle6layers6rotary11get_cos_sin17h5874782181534d74E+0x2f54): undefined reference to `vdCos'
          /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-8ff1bbf6461b1988.text_embeddings_router.bb30d0383363c7cb-cgu.0.rcgu.o: in function `<candle_core::cpu_backend::CpuStorage as candle_core::backend::BackendStorage>::binary_impl':
          text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17hcd10a387af03f060E+0x831d): undefined reference to `vsFmin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17hcd10a387af03f060E+0x8bc4): undefined reference to `vsFmin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17hcd10a387af03f060E+0x8d43): undefined reference to `vsFmin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17hcd10a387af03f060E+0x9c59): undefined reference to `vdFmin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17hcd10a387af03f060E+0xa604): undefined reference to `vdFmin'
          /usr/bin/ld: text_embeddings_router.bb30d0383363c7cb-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$11binary_impl17hcd10a387af03f060E+0xa777): undefined reference to `vdFmin'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `text-embeddings-router` (bin "text-embeddings-router") due to 1 previous error
error: failed to compile `text-embeddings-router v1.8.2 (/home/usr1/text-embeddings-inference-1.8.2/router)`, intermediate artifacts can be found at `/home/usr1/text-embeddings-inference-1.8.2/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

intel-mkl-tool crate seeks system MKL library installed by various installer as following manner:

  • Seek using pkg-config command
  • Seek ${MKLROOT} directory

If intel-mkl-tool does not find MKL library, intel-mkl-src try to download MKL binaries from GitHub Container Registry (ghcr.io).

Seems it's not trying to link a local MKL binary, nor download a useful MKL binaries?

PS: I am using Qwen3-embedding-4B

nkh0472 avatar Sep 25 '25 13:09 nkh0472

Hmm so that's odd since given that the default is dynamic-linking it should indeed try to dynamically link the Intel MKL oneAPI libraries... Let me investigate further, but after the fix everything should work as expected on Intel CPUs, is the cargo install a blocker for you? May I suggest you use the ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.2 image in the meantime?

And as per Qwen3-Embedding-4B there's not an export for in under https://huggingface.co/onnx-community, so you can export it yourself using any of the aforementioned solutions, then --feature ort should do the work!

Again, let me investigate the dynamic-linking issue further and let you know, did you try either setting the LD_LIBRARY_PATH environment variable or using static-linking instead as proposed in earlier comments?

Thanks and apologies for the inconvenience 🙏🏻

alvarobartt avatar Sep 25 '25 14:09 alvarobartt

Again, let me investigate the dynamic-linking issue further and let you know, did you try either setting the LD_LIBRARY_PATH environment variable or using static-linking instead as proposed in earlier comments?

LD_LIBRARY_PATH is not helpful.

> cargo install --path router --features candle,mkl,static-linking,http --no-default-features --verbose
...
Running `/home/usr1/text-embeddings-inference-1.8.2/target/release/build/intel-mkl-src-4d145e8db00d0730/build-script-build`
    Building [=======================> ] 594/606: intel-mkl-src(build)

It stuck at here for more than 1 hours, I'll check it tomorrow.

Hmm so that's odd since given that the default is dynamic-linking it should indeed try to dynamically link the Intel MKL oneAPI libraries... Let me investigate further, but after the fix everything should work as expected on Intel CPUs, is the cargo install a blocker for you? May I suggest you use the ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.2 image in the meantime?

I have download TEI docker image on Windows env: docker run -itd --name TEI-Qwen3-emb-4B -p 8851:80 -v E:\.cache\modelscope\hub\models\Qwen\:/data ghcr.nju.edu.cn/huggingface/text-embeddings-inference:cpu-latest --model-id /data/Qwen3-Embedding-4B

But I'm not sure why it cannot load original huggingface model (does this docker image also using ONNX backend?):

2025-09-25 23:18:50.581 | 2025-09-25T15:18:50.579112Z  INFO text_embeddings_router: router/src/main.rs:203: Args { model_id: "/dat*/*****-********g-4B/", revision: None, tokenization_workers: None, dtype: None, pooling: None, max_concurrent_requests: 512, max_batch_tokens: 16384, max_batch_requests: None, max_client_batch_size: 32, auto_truncate: false, default_prompt_name: None, default_prompt: None, dense_path: None, hf_api_token: None, hf_token: None, hostname: "b558c268c474", port: 80, uds_path: "/tmp/text-embeddings-inference-server", huggingface_hub_cache: Some("/data"), payload_limit: 2000000, api_key: None, json_output: false, disable_spans: false, otlp_endpoint: None, otlp_service_name: "text-embeddings-inference.server", prometheus_port: 9000, cors_allow_origin: None }
2025-09-25 23:18:50.835 | 2025-09-25T15:18:50.835538Z  WARN text_embeddings_router: router/src/lib.rs:190: Could not find a Sentence Transformers config
2025-09-25 23:18:50.835 | 2025-09-25T15:18:50.835565Z  INFO text_embeddings_router: router/src/lib.rs:194: Maximum number of tokens per request: 40960
2025-09-25 23:18:50.837 | 2025-09-25T15:18:50.837035Z  INFO text_embeddings_core::tokenization: core/src/tokenization.rs:38: Starting 16 tokenization workers
2025-09-25 23:18:51.238 | 2025-09-25T15:18:51.238561Z  INFO text_embeddings_router: router/src/lib.rs:242: Starting model backend
2025-09-25 23:18:51.246 | 2025-09-25T15:18:51.245902Z ERROR text_embeddings_backend: backends/src/lib.rs:402: Could not start ORT backend: Could not start backend: File at `/data/Qwen3-Embedding-4B/onnx/model.onnx` does not exist
2025-09-25 23:18:51.262 | 2025-09-25T15:18:51.262159Z  INFO text_embeddings_backend_candle: backends/candle/src/lib.rs:305: Starting Qwen3 model on Cpu

root@b558c268c474:/# ls /data/Qwen3-Embedding-4B/
1_Pooling  config.json                        configuration.json      merges.txt                        model-00002-of-00002.safetensors  modules.json    tokenizer_config.json
README.md  config_sentence_transformers.json  generation_config.json  model-00001-of-00002.safetensors  model.safetensors.index.json      tokenizer.json  vocab.json

Meanwhile, I have some problem on converting the model to onnx, but it's not related to this issue, and I don't it'll be a big problem.

❯ uv pip install 'optimum[exporters]'
❯ optimum-cli export onnx --model ./.cache/modelscope/hub/models/Qwen/Qwen3-Embedding-4B/ ./Qwen3-Embedding-4B-onnx --task feature-extraction
/home/usr1/.local/intelpython3/envs/vllm/lib/python3.12/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
Framework not specified. Using pt to export the model.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 2/2 [00:01<00:00,  1.45it/s]
Using the export variant default. Available variants are:
    - default: The default ONNX variant.
Using framework PyTorch: 2.8.0+cu129
Overriding 1 configuration item(s)
        - use_cache -> False
/home/usr1/.local/intelpython3/envs/vllm/lib/python3.12/site-packages/transformers/masking_utils.py:207: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if (padding_length := kv_length + kv_offset - attention_mask.shape[-1]) > 0:
/home/usr1/.local/intelpython3/envs/vllm/lib/python3.12/site-packages/transformers/masking_utils.py:235: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if padding_mask is not None and padding_mask.shape[-1] > kv_length:
Traceback (most recent call last):
...
RuntimeError: unordered_map::at

I also tried modifying https://github.com/randomm/qwen3-embedding-tei-onnx/blob/main/convert_qwen3_tei_onnx.py to convert my local 4B model, gave same error:

/tmp/ipykernel_3031144/1821030884.py:114: DeprecationWarning: You are using the legacy TorchScript-based ONNX export. Starting in PyTorch 2.9, the new torch.export-based ONNX exporter will be the default. To switch now, set dynamo=True in torch.onnx.export. This new exporter supports features like exporting LLMs with DynamicCache. We encourage you to try it and share feedback to help improve the experience. Learn more about the new export logic: https://pytorch.org/docs/stable/onnx_dynamo.html. For exporting control flow: https://pytorch.org/tutorials/beginner/onnx/export_control_flow_model_to_onnx_tutorial.html.
  torch.onnx.export(
/home/usr1/.local/intelpython3/envs/vllm/lib/python3.12/site-packages/transformers/masking_utils.py:207: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if (padding_length := kv_length + kv_offset - attention_mask.shape[-1]) > 0:
/home/usr1/.local/intelpython3/envs/vllm/lib/python3.12/site-packages/transformers/masking_utils.py:235: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if padding_mask is not None and padding_mask.shape[-1] > kv_length:
...

Anyway, I'll try zhiqing/Qwen3-Embedding-4B-ONNX tomorrow.

nkh0472 avatar Sep 25 '25 15:09 nkh0472

For using zhiqing/Qwen3-Embedding-4B-ONNX, I have to manual copy 1_Pooling from Qwen/Qwen3-Embedding-4B.

Then both windows build and docker:cpu-latest gives error:

Error: Model backend is not healthy

Caused by:
    Cannot extract Tensor<f32> from Tensor<f16>

:(

nkh0472 avatar Sep 25 '25 16:09 nkh0472

Hey @nkh0472 indeed as docker run seems to be unblocking you that's nice, but let's clarify the following points:

  • The Dockerfile for CPU comes with both ort and mkl features
  • ONNX Runtime aka ort is used as the first option, but if the ONNX weights are not within the repository, then candle + mkl is used as the fallback (which is the case for e.g. Qwen/Qwen3-Embedding-4B)
  • By default ONNX export (unless part of the model with the default Safetensor weights) don't contain the 1_Pooling/ directory, meaning that when running ONNX models you should provide the --pooling argument with it's Safetensors counterpart e.g. Qwen/Qwen3-Embedding-0.6B uses last-token pooling hence when running onnx-community/Qwen3-Embedding-0.6B-ONNX you should set the argument --pooling last-token (otherwise it will complain)

This being said the aforementioned errors are expected and your command should look something like the following:

docker run --platform linux/amd64 -p 8080:80 ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.2 --model-id onnx-community/Qwen3-Embedding-0.6B-ONNX --dtype float32 --pooling last-token

As per the ONNX export, let me look into it and share it with you once exported; and as per the issue when installing with mkl with dynamic-linking I'll look into it and see if I can reproduce 🤗

alvarobartt avatar Sep 25 '25 17:09 alvarobartt

This being said the aforementioned errors are expected and your command should look something like the following:

docker run --platform linux/amd64 -p 8080:80 ghcr.io/huggingface/text-embeddings-inference:cpu-1.8.2 --model-id onnx-community/Qwen3-Embedding-0.6B-ONNX --dtype float32 --pooling last-token

Both docker image and windows build could load onnx-community/Qwen3-Embedding-0.6B-ONNX properly:

docker run -itd --name TEI-Qwen3-emb-0.6B -v E:\.cache\modelscope\hub\models\onnx-community\Qwen3-Embedding-0___6B-ONNX:/data -p 8851:80 ghcr.nju.edu.cn/huggingface/text-embeddings-inference:cpu-latest --model-id /data --dtype float32 --pooling last-token --tokenization-workers 1 --max-batch-tokens 8192 --max-batch-requests 5 --max-client-batch-size 1 --max-concurrent-requests 5

text-embeddings-router.exe -p 8851 --model-id E:\.cache\modelscope\hub\models\onnx-community\Qwen3-Embedding-0___6B-ONNX --dtype float32 --pooling last-token --tokenization-workers 1 --max-batch-tokens 8192 --max-batch-requests 5 --max-client-batch-size 1 --max-concurrent-requests 5

And Qwen/Qwen3-Embedding-4B successfully started in docker image, with some errors from MKL (I didn't check if the value of embeddings is correct):

docker run -itd --name TEI-Qwen3-emb-4B-raw -v E:\.cache\modelscope\hub\models\Qwen\Qwen3-Embedding-4B:/data -p 8851:80 ghcr.nju.edu.cn/huggingface/text-embeddings-inference:cpu-latest --model-id /data --pooling last-token --tokenization-workers 1 --max-batch-tokens 8192 --max-batch-requests 5 --max-client-batch-size 1 --max-concurrent-requests 5

2025-09-26 01:56:57.098 | 2025-09-25T17:56:57.095477Z  INFO text_embeddings_router: router/src/main.rs:203: Args { model_id: "/****", revision: None, tokenization_workers: Some(1), dtype: None, pooling: Some(LastToken), max_concurrent_requests: 5, max_batch_tokens: 8192, max_batch_requests: Some(5), max_client_batch_size: 1, auto_truncate: false, default_prompt_name: None, default_prompt: None, dense_path: None, hf_api_token: None, hf_token: None, hostname: "fa7aabdcaaa5", port: 80, uds_path: "/tmp/text-embeddings-inference-server", huggingface_hub_cache: Some("/data"), payload_limit: 2000000, api_key: None, json_output: false, disable_spans: false, otlp_endpoint: None, otlp_service_name: "text-embeddings-inference.server", prometheus_port: 9000, cors_allow_origin: None }
2025-09-26 01:56:57.464 | 2025-09-25T17:56:57.463882Z  WARN text_embeddings_router: router/src/lib.rs:190: Could not find a Sentence Transformers config
2025-09-26 01:56:57.464 | 2025-09-25T17:56:57.463914Z  INFO text_embeddings_router: router/src/lib.rs:194: Maximum number of tokens per request: 40960
2025-09-26 01:56:57.465 | 2025-09-25T17:56:57.465034Z  INFO text_embeddings_core::tokenization: core/src/tokenization.rs:38: Starting 1 tokenization workers
2025-09-26 01:56:57.513 | 2025-09-25T17:56:57.512914Z  INFO text_embeddings_router: router/src/lib.rs:242: Starting model backend
2025-09-26 01:56:57.518 | 2025-09-25T17:56:57.518035Z ERROR text_embeddings_backend: backends/src/lib.rs:402: Could not start ORT backend: Could not start backend: File at `/data/onnx/model.onnx` does not exist
2025-09-26 01:56:57.589 | 2025-09-25T17:56:57.588896Z  INFO text_embeddings_backend_candle: backends/candle/src/lib.rs:305: Starting Qwen3 model on Cpu
2025-09-26 02:03:57.998 | 2025-09-25T18:03:57.998533Z  INFO text_embeddings_router: router/src/lib.rs:260: Warming up model
2025-09-26 02:04:04.995 | 
2025-09-26 02:04:04.995 | Intel MKL ERROR: Parameter 1 was incorrect on entry to vsAdd.

(skip other 35 same errors)

2025-09-26 02:10:44.887 | 
2025-09-26 02:10:44.887 | Intel MKL ERROR: Parameter 1 was incorrect on entry to vsAdd.
2025-09-26 02:10:51.911 | 2025-09-25T18:10:51.908982Z  WARN text_embeddings_router: router/src/lib.rs:269: Backend does not support a batch size > 4
2025-09-26 02:10:51.911 | 2025-09-25T18:10:51.911449Z  WARN text_embeddings_router: router/src/lib.rs:270: forcing `max_batch_requests=4`
2025-09-26 02:10:51.919 | 2025-09-25T18:10:51.918852Z  WARN text_embeddings_router: router/src/lib.rs:319: Invalid hostname, defaulting to 0.0.0.0
2025-09-26 02:10:52.108 | 2025-09-25T18:10:52.107860Z  INFO text_embeddings_router::http::server: router/src/http/server.rs:1852: Starting HTTP server: 0.0.0.0:80
2025-09-26 02:10:52.108 | 2025-09-25T18:10:52.107881Z  INFO text_embeddings_router::http::server: router/src/http/server.rs:1853: Ready
2025-09-26 02:19:31.712 | 2025-09-25T18:19:31.712168Z  INFO embed{total_time="3.819215749s" tokenization_time="26.132586ms" queue_time="1.014985ms" inference_time="3.791110111s"}: text_embeddings_router::http::server: router/src/http/server.rs:733: Success
Image

I think I need to give up in this way due to both bad performance and potential value error.

> cargo install --path router --features candle,mkl,static-linking,http --no-default-features --verbose
...
Running `/home/usr1/text-embeddings-inference-1.8.2/target/release/build/intel-mkl-src-4d145e8db00d0730/build-script-build`
    Building [=======================> ] 594/606: intel-mkl-src(build)

It stuck at here for more than 1 hours, I'll check it tomorrow.

New error shows after a long time compiling:

Running `/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/bin/rustc --crate-name text_embeddings_router --edition=2021 router/src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=125 --crate-type bin --emit=dep-info,link -C opt-level=3 -C panic=abort -C lto=fat -C codegen-units=1 --cfg 'feature="candle"' --cfg 'feature="http"' --cfg 'feature="mkl"' --cfg 'feature="static-linking"' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values("accelerate", "candle", "candle-cuda", "candle-cuda-turing", "candle-cuda-volta", "default", "dynamic-linking", "google", "grpc", "http", "metal", "mkl", "ort", "python", "static-linking"))' -C metadata=d5fe77ef25762be6 -C extra-filename=-2566ce3fc6947516 --out-dir /home/usr1/text-embeddings-inference-1.8.2/target/release/deps -C strip=symbols -L dependency=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps --extern anyhow=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libanyhow-4660ba28df6f7c9a.rlib --extern axum=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libaxum-4546ad3d65e14eb4.rlib --extern axum_tracing_opentelemetry=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libaxum_tracing_opentelemetry-deb860be745bc1e2.rlib --extern base64=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libbase64-45c66f817e4078b8.rlib --extern clap=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libclap-a8bdfbe4b8b01368.rlib --extern futures=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libfutures-c1f5e7dabacfaf8d.rlib --extern hf_hub=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libhf_hub-16fbae865bccb279.rlib --extern http=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libhttp-1e1220ba9590e870.rlib --extern init_tracing_opentelemetry=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libinit_tracing_opentelemetry-f21999f6202a6396.rlib --extern intel_mkl_src=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libintel_mkl_src-6b1bea82e24d9f57.rlib --extern libc=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/liblibc-0a7b01598885b7e1.rlib --extern metrics=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libmetrics-5231a4984ec5dbc6.rlib --extern metrics_exporter_prometheus=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libmetrics_exporter_prometheus-809e3beb38e1782e.rlib --extern num_cpus=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libnum_cpus-860516a882c060a0.rlib --extern opentelemetry=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libopentelemetry-2bfa186370b15a82.rlib --extern opentelemetry_otlp=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libopentelemetry_otlp-511cbf05fac3bdd2.rlib --extern opentelemetry_sdk=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libopentelemetry_sdk-8c0f54e4e58a99fe.rlib --extern reqwest=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libreqwest-c77a74b4ac92cd57.rlib --extern serde=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libserde-142cc40a82007987.rlib --extern serde_json=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libserde_json-666cb6b47eabac68.rlib --extern simsimd=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libsimsimd-e642e88b7c150607.rlib --extern text_embeddings_backend=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtext_embeddings_backend-824b626f9e1c0cb7.rlib --extern text_embeddings_core=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtext_embeddings_core-6184cca7678f690f.rlib --extern text_embeddings_router=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtext_embeddings_router-4f0a68c5b2ae5f7d.rlib --extern thiserror=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libthiserror-4d538c63670ab48b.rlib --extern tokenizers=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtokenizers-59eff9b998c3588d.rlib --extern tokio=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtokio-c09adedaec5226f9.rlib --extern tower_http=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtower_http-4963fb0c66f4eb86.rlib --extern tracing=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtracing-5e56e748ade71738.rlib --extern tracing_opentelemetry=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtracing_opentelemetry-b0a2b2e4fa4a1069.rlib --extern tracing_subscriber=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libtracing_subscriber-15b1d6722153a745.rlib --extern utoipa=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libutoipa-665888d65dd6f056.rlib --extern utoipa_swagger_ui=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libutoipa_swagger_ui-98aa3a87580c0844.rlib --extern veil=/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/libveil-59cfab4f390256bb.rlib -C target-cpu=native -L native=/home/usr1/.local/lib64 -L /home/usr1/.local/share/ocipkg/ghcr.io/rust-math/rust-mkl/linux/mkl-static-lp64-iomp/__2020.1-3038006115 -L native=/home/usr1/text-embeddings-inference-1.8.2/target/release/build/aws-lc-sys-36da19d0f9c47d3b/out -L native=/home/usr1/text-embeddings-inference-1.8.2/target/release/build/simsimd-08e34d61368bf0c3/out -L native=/home/usr1/text-embeddings-inference-1.8.2/target/release/build/esaxx-rs-b7b08fd242b53a8e/out -L native=/home/usr1/text-embeddings-inference-1.8.2/target/release/build/onig_sys-e7ba94824b04f014/out`
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/home/usr1/.cargo/bin:/home/usr1/.local/intelpython3/envs/vllm/bin:/home/usr1/.local/intelpython3/condabin:/home/usr1/.local/gcc-10/bin:/home/usr1/.local/openssl/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin:/home/usr1/.local/bin:/home/usr1/.local/cuda-12.2/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustcMSjfuh/symbols.o" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/tmp/rustcMSjfuh/{libesaxx_rs-e75533045cd99d2f.rlib,libonig_sys-e47cc43c9997aa8d.rlib,libsimsimd-e642e88b7c150607.rlib,libaws_lc_sys-298ab5103f7b8529.rlib,libintel_mkl_src-6b1bea82e24d9f57.rlib}" "/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libcompiler_builtins-6497188c539eb018.rlib}" "-Wl,-Bdynamic" "-lstdc++" "-ldl" "-lssl" "-lcrypto" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/usr1/.local/lib64" "-L" "/home/usr1/.local/share/ocipkg/ghcr.io/rust-math/rust-mkl/linux/mkl-static-lp64-iomp/__2020.1-3038006115" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/aws-lc-sys-36da19d0f9c47d3b/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/simsimd-08e34d61368bf0c3/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/esaxx-rs-b7b08fd242b53a8e/out" "-L" "/home/usr1/text-embeddings-inference-1.8.2/target/release/build/onig_sys-e7ba94824b04f014/out" "-L" "/home/usr1/.rustup/toolchains/1.85.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-2566ce3fc6947516" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: /usr/bin/ld: /home/usr1/text-embeddings-inference-1.8.2/target/release/deps/text_embeddings_router-2566ce3fc6947516.text_embeddings_router.e7264c00e38c959a-cgu.0.rcgu.o: in function `<candle_core::cpu_backend::CpuStorage as candle_core::backend::BackendStorage>::matmul':
          text_embeddings_router.e7264c00e38c959a-cgu.0:(.text._ZN93_$LT$candle_core..cpu_backend..CpuStorage$u20$as$u20$candle_core..backend..BackendStorage$GT$6matmul17h3bffd05811984420E+0x179b): undefined reference to `hgemm_'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `text-embeddings-router` (bin "text-embeddings-router") due to 1 previous error

nkh0472 avatar Sep 25 '25 18:09 nkh0472

So it's clear that the error is with the linking, as it fails when compiling the project with mkl complaining due to e.g. "error: linking with cc failed: exit status: 1" so that means that the binary is not able to link Intel MKL oneAPI during compilation... Let me investigate further, but so far I've been unable to reproduce, if you could share how you installed the Intel MKL libraries that'd be helpful, as most likely the error is around that...

Also the error you reported above as "Intel MKL ERROR: Parameter 1 was incorrect on entry vsAdd" is due to the Intel MKL optimizations, let me see if I can reproduce that one and see if the issue is in the underlying CPU backend for candle with Intel MKL, or something else instead.

alvarobartt avatar Sep 26 '25 15:09 alvarobartt

I think I had figure out how to reproduce it.

    1.use an environment without mkl
    2.compile the code
    3.set the mkl env var
    4.compile (this problem happend)

How to fix? use cargo clean perhaps it was cause by some cargo cache issue. BTW,please set the env by using source /opt/intel/oneapi/setvars.sh

meipeter avatar Nov 18 '25 14:11 meipeter