Metal error with Flux image diffusion cast_bf16_f32 does not exist

[AlpineVibrations]

Running the new Flux model on Mac M1 Metal

running the example

 cargo run --features metal --example flux -r -- \
    --height 1024 --width 1024 \
    --prompt "a rusty robot walking on a beach holding a small torch, the robot has the word "rust" written on it, high quality, 4k" --model schnell

and after downloading the models and trying to run

Tensor[[1, 768], f32, metal:4294969333]
Error: Metal error Error while loading function: "Function 'cast_bf16_f32' does not exist"

Caused by:
    Error while loading function: "Function 'cast_bf16_f32' does not exist"
    

[jk2K]

related to https://github.com/huggingface/candle/issues/2163

[LaurentMazare]

Just to mention that I was able to run flux on a macbook pro M3 Max earlier this week. I don't think this explains the problem you encountered but prior to this week we were missing the bf16 matmul kernels which are now available. That said the error about missing cast_bf16_f32 seems more like using a clang toolchain that doesn't support bfloat, e.g. clang 14, or that the hardware itself might not support bfloat. Some related issue for tinygrad: https://github.com/tinygrad/tinygrad/issues/3453

[AlpineVibrations]

Great to hear. thanks. We just tried and it still errors out with the same error on M1 and M3 with metal.

[AlpineVibrations]

checking in on this again. so the idea is that maybe clang is wrong version? just tried again with latest candle and still same error

function 'cast_f32_bf16' does not exist