Unable to compile candle-core under Raspberry Pi

[dayn9t]

cat /etc/issue Debian GNU/Linux 12 \n \l

rustc -V rustc 1.77.0-nightly (d5fd09972 2024-01-22)

gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/12/lto-wrapper Target: aarch64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Debian 12.2.0-14' --with-bugurl=file:///usr/share/doc/gcc-12/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-12 --program-prefix=aarch64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 12.2.0 (Debian 12.2.0-14)

The error message is as follows:

error: instruction requires: fullfp16 --> /home/jiang/.cargo/registry/src/mirrors.sjtug.sjtu.edu.cn-4f7dbcce21e258a2/gemm-common-0.17.0/src/simd.rs:1982:18 | 1982 | "fmla {0:v}.8h, {1:v}.8h, {2:v}.h[0]", | ^ |

[EricLBuehler]

Are you attempting to compile with CUDA, flash attention or anything which requires CUDA kernels?

[sarah-quinones]

this looks like a bug on my part. will try to fix it soon-ish

[sarah-quinones]

this might be a rustc bug actually, there's no feature called fullfp16 on the rust side

[Pox-here]

Experiencing the same. Tried to follow the thread further but found currently no solution. Was this issue solved?

    --> /home/[user]/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gemm-common-0.17.1/src/simd.rs:2024:18
     |
2024 |                 "fmla {0:v}.8h, {1:v}.8h, {2:v}.h[7]",
     |                  ^
     |
note: instantiated into assembly here
    --> <inline asm>:1:2
     |
1    |     fmla v0.8h, v1.8h, v2.h[7]

Tnx

[evilsocket]

Can confirm this happens for any ARM target (in my case aarch64-apple-ios) even without default features enabled. I suspect the reason is this not being set as optional/conditional. As far as I can tell gemm does not support ARM.

[evilsocket]

Solved by adding this to my .cargo/config.toml:

[target.'cfg(any(target_arch = "arm", target_arch = "aarch64"))']
rustflags = ["-C", "target-feature=+fp16"]

[gauthamk28]

Solved by adding this to my .cargo/config.toml:

[target.'cfg(any(target_arch = "arm", target_arch = "aarch64"))']
rustflags = ["-C", "target-feature=+fp16"]

After adding this in my raspberry pi 5 its still not working , can you please help me fix this ?

[AlpineVibrations]

getting the same error trying to compile for iOS tried the rustflags solution above and it is still not working.

[skyne98]

Same here!

Compiling on Windows for Android. Rustflags are set. Config is set to the Android NDK.

Is there a solution to this?

@sarah-ek

Thanks for the help in advance :)

[sarah-quinones]

dunno

[AlpineVibrations]

removing gemm dependency for iOS got it to build.

[gauthamk28]

removing gemm dependency for iOS got it to build.

Can you please tell me how did you got it to build , because when I removed gemm dependancy , I am getting the below error: gemm is used in candle-core/src/cpu_backend/mod.rs:1256:13

[zackangelo]

Solved by adding this to my .cargo/config.toml:

[target.'cfg(any(target_arch = "arm", target_arch = "aarch64"))'] rustflags = ["-C", "target-feature=+fp16"]

Was also running into this when trying to compile on a GH200 (Nvidia ARM chip).

Funnily enough, it worked when compiling in release mode without this flag. It was only when I tried to compile in debug mode that the error happened.

Adding that flag fixed it for me, though.

[Ch4s3r]

I was using candle for an Android app with dioxus and found two options for this error:

17.354s  INFO  error: instruction requires: fullfp16
...
17.381s ERROR error: could not compile `gemm-f16` (lib) due to 11 previous errors 

Solutions:

• Build gemm-f16 with opt-level = 3 for debug builds

  profile.dev.package.gemm-f16]
  pt-level = 3
  

• Or run in release mode

[siennathesane]

Here is a workaround: https://github.com/sarah-quinones/gemm/issues/31