joshpopelka20
joshpopelka20
> Essentially you need to split Block into a few steps. Can you give me more details on the steps? That's where I'm confused. > I would also recommend you...
I did a simple test to the mapper function that you pointed out above. I've confirmed that it is the problem. Here is the code snippet of the test: ...
I don't want this to fall to the wayside, as I'll need it for my long context usecase. 1. The latest commit has the sequence parallelism code using IndexOp. I...
@EricLBuehler thanks for adding this feature. When using the pypi package, I'm getting this error: ``` = note: /usr/bin/ld: /tmp/pip-install-8y1wtzkm/mistralrs-cuda_36180abfef7b4f0687d7842e9b298d9e/target/release/build/mistralrs-core-f5a6ed7a31cbdb62/out/libmistralcuda.a(nonzero_bitwise-b50867152df76f01.o): relocation R_X86_64_32 against symbol `_Z17transform_indicesPKjjS0_jPj' can not be used when...
I'm using a Sagemaker Jupyter notebook so Amazon Linux. This is the distro info: ``` NAME="Amazon Linux" VERSION="2" ID="amzn" ID_LIKE="centos rhel fedora" VERSION_ID="2" PRETTY_NAME="Amazon Linux 2" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" HOME_URL="https://amazonlinux.com/" SUPPORT_END="2025-06-30"...
Adding some observations: When I run `CUDA_NVCC_FLAGS=-fPIE cargo build --release --features "cuda cudnn"` and `CUDA_NVCC_FLAGS=-fPIE cargo build --release --features "cuda flash-attn"`, I don't get the error. When I run `CUDA_NVCC_FLAGS=-fPIE...
I'm thinking the issue is related to mistralrs-core/build.rs file. I tried to add ``` .arg("--compiler-options") .arg("-fPIC") ``` but it didn't help. I think it's more of an issue with this...
I added a pull request with a fix for the issue [https://github.com/EricLBuehler/mistral.rs/pull/471](https://github.com/EricLBuehler/mistral.rs/pull/471). Looks like it was a divide by zero issue. I didn't add any error message; I just let...
The layers are now distributed across my 4 A10G GPUs  One request I have is: can there be a progress bar added when the model is being loaded? For...
There were no issues building from source. Also, the 2 day delay is not an issue for me. Finally, I think the ask for this issue is complete. Would you...