candle
candle copied to clipboard
huggingface
Minimalist ML framework for Rust
Hey there! I'm reaching out from a machine running the Intel i3-2120 processor. The `llm-rs` library (https://github.com/rustformers/llm) can handle inference just fine on this processor (previously failed on older CPUs,...
I am trying to use `topk` to implement X-LoRA in Candle, and want to perform `topk` in the last dimension. Specifically, I need the `indices` return value (as returned by...
After running multiple times the command: ``` cargo run --release --features metal --example phi -- --model 3 --prompt "The best thing about coding in rust is " ``` I realized...
Hello, running q80 quantized whisper on Android (Pixel 7) is taking around 15 seconds for 5 seconds of audio. Is there any way to speed this up that I might...
I am relatively new so I hope I am not just doing something very stupid :) I am trying to adapt the [quantized example](https://github.com/mokeyish/candle/blob/14a3fb0b33e9175f50f3ba47e8dbcde5ce6e4ceb/candle-examples/examples/quantized/main.rs) for my use case. The inference...
Do you have any plans to support multimodal LLMs, such as MiniGPT-4/MiniGPT v2 (https://github.com/Vision-CAIR/MiniGPT-4/) and LLaVA (https://github.com/haotian-liu/LLaVA/)? That would be a significant enhancement if these popular multimodal LLMs were supported...
I train one model by yolo8, and export to a onnx file, then import by candel-onnx . An error accur: `Error: unsupported op_type Split for op NodeProto { input: ["/model.2/cv1/act/Mul_output_0",...
Please allow to use `microsoft/Phi-3-mini-128k-instruct` model in the [candle-phi](https://github.com/huggingface/candle/tree/main/candle-examples/examples/phi) example, which uses the LongRope scaling. Thanks!
``` let a = Tensor::new(&[3u8], &candle::Device::Cpu)?; let b = a / 4f64; ``` What's the value of tensor `b`? Wow, it's `Ok(Tensor[0; u8])`. Rust compiler rejects code like `3u8 /...
The current project is based on YOLOV5，Pytorch. We are required to use rust/wasm to do the same thing（Reuse existing Pytorch model）. Now It's already supported YOLOV8，so Will candle support YOLOV5？...
