Eric Buehler
Eric Buehler
@ivanstepanovftw `Tensor::matmul` allows batching support, similar to `torch.bmm`. Something like: ```rust use candle_core::Device::Cpu; use candle_core::Tensor; use std::time::Instant; fn cpu_test() { // Create two random tensors on the CPU let a...
@vasileermicioi could this be due to an enviornment configuration error? Perhaps the nvcc and nvidia-smi versions are different, nvcc should be less than nvidia-smi.
@ShelbyJenkins can you reproduce the issue if you ensure running without PagedAttention?
@ShelbyJenkins I'll take a look at what is causing this.
@ShelbyJenkins I cannot reproduce the error, please feel free to reopen.
@misureaudio @gfxenjoyer I pushed a fix in #859, can you please confirm and I can merge?
@misureaudio I merged #859!
@misureaudio closing, please feel free to reopen!
> @EricLBuehler with these schemas added, that just leaves `Either` and [`LlguidanceGrammar`](https://github.com/EricLBuehler/mistral.rs/blob/e40947e449cd59d4868b3d858d13b1681d480fb7/mistralrs-core/src/request.rs#L16). > > ``` > Resolver error at paths./v1/chat/completions.post.requestBody.content.application/json.schema.properties.grammar.oneOf.2.properties.value.$ref > Could not resolve reference: Could not resolve pointer: /components/schemas/LlguidanceGrammar...
@matthewhaynesonline that looks good for `Either`. Let me know when this is ready for review!