candle
candle copied to clipboard
Enabling different linking options for MKL downstream.
When using candle/mkl right now, it forces the use of static linking.
However, this is not optimal in some circumstances, for instance we need to hotpatch the library to enable faster runtime on non-Intel CPU: https://github.com/huggingface/text-embeddings-inference/blob/main/Dockerfile#L39-L40
In order for the thing to be practical it's easier to use dynamic linking and use clever ordering to force patch the library at runtime.
I made the change non breaking.
- feature
mklbecomes_mkland doesn't include the actualintel-mkl-src/mkl-static-lp64-iompfeature, so it's agnostic the linking procedure. - Adding a new
mklfeature which solely addsmkl-static-lp64-iompto keep backward compatibility in examples and downstream crates - Propagate to
candle-nn - Didn't propagate to
candle-transformers(user can simply opt-out ofmklaltogether since no code actually uses it)
So now for power-users than want to custom link mkl all they need to do is enable the _mkl feature, and manually enable the intel-mkl-src/mkl-dynamic-lp64-iomp in their own crate/binary.
I haven't modified the documentation since it's a power user feature anyway, but I can do it if deemed necessary.