candle
candle copied to clipboard
huggingface
Minimalist ML framework for Rust
I am trying to infer mlx-community/gemma-2-2b-it-4bit using the candle-examples/gemma sample. -------------------------- But the following error occurs: Error: shape mismatch for model.embed_tokens.weight, expected: [256000, 2304], got: [256000, 288] It's very difficult...
This change implements an new matmul based on Armv8.6 i8mm instructions. As the change requires a nightly compiler, using the more performant version is optional, under the new `arm-nightly-feat` feature...
hi. gettin an error. on my ADA RTX 4000 machine that supports BF16 and that runs Stable Diffusion just fine. I get an error on the quantized FLUX update. running...
For running Models please add vulkan backend, pytorch has this and you can use wgpu for that too.
I was unable to use [clip-vit-base-patch16](https://huggingface.co/openai/clip-vit-base-patch16) as there is no deserialization available for the CLIP Configuration. The only model that can be used is clip-vit-base-patch-32. In PR #2527 I have...
Earlier, it was not possible to use a Clip model by providing a model ID and revision because the ClipConfig was not loaded from the config.json file. I made the...
Hi, I have a working implementation of [Stella_en__v5](https://huggingface.co/dunzhang/stella_en_1.5B_v5) family of models which is one of the top ranking model in the MTEB leaderboard for reranking and retrieval. It's basically built...
Does Candle have any plans to support ARM Kleidi in the future?
Running executable with candle on Ubuntu 22 we get an error when trying to generate an image ```DriverError(CUDA_ERROR_UNSUPPORTED_PTX_VERSION, \"the provided PTX was compiled with an unsupported toolchain.\") when loading is_u32_f32```...
