Mimi Encoder

[ArEnSc]

It seems like Mimi doesn't work on Cuda and using it via cpu requires 1 second to transcribe 80 ms of audio. Why is this the case? Thanks. It did mention something along the lines for Cuda there is an missing cuda implementation for rot emb i not implemented.

[LaurentMazare]

Mimi should work fine on cuda (I just double checked and it was all ok). Could you provide some specific error? From what you mention maybe you forgot to activate the cuda feature on the candle-nn crate (you have to activate it both on candle-core and candle-nn).

[ArEnSc]

@LaurentMazare you were right I fixed it! is there a way to get mimi to work on a cpu at a reasonable rate as well? I suspect this just might be feature flags?

[LaurentMazare]

Depend what you call "reasonable rate", mimi should be able to do real-time encoding and decoding on many cpus. You will want to compile in release mode and ideally use compiler option target-cpu=native so as to enable simd acceleration. You can use the following snippet to check if simd extensions were properly detected (you should see the neon one on apple silicon and avx on x86).

    println!(
        "avx: {}, neon: {}, simd128: {}, f16c: {}",
        candle::utils::with_avx(),
        candle::utils::with_neon(),
        candle::utils::with_simd128(),
        candle::utils::with_f16c()
    );