Henry Kroll III

This issue WAS happening, before totally wiping out the system and upgrading fresh on a new hard drive. This time installing updated video drivers, cuda, and cudnn from the nvidia...

I could not install torch in the same venv as whisper-jax when making my open-source [Whisper Dictation](https://github.com/themanyone/whisper_dictation) app here. Doing so would downgrade nvidia-cudnn-cu11 to a non-working version that would...

I got cuda-cupti package for Fedora. Ubuntu should have something similar. It came with the cuda 11-7 repo install from the nvidia website.

Oh, I also had to set some environment in .bashrc to get it to recognise. ``` export CUDA_HOME=/usr/local/cuda export LIBRARY_PATH="${CUDA_HOME}/lib64" export C_INCLUDE_PATH="${CUDA_HOME}/include" ``` Because everything should be an easy install...

I was able to make it work. You don't get speedup on the first transcription, which is slower due to compile. You get better performance on subsequent calls to the...

> JAX produces inaccurate transcription compared to the original OpenAI with a lot of repeating of the words and gibberish something to do with the hallucination issue... We don't get...

ChatGPT says it should work if you load the model from huggingface, and then perform conversion of "model": ``` model = AutoModelForSeq2SeqLM.from_pretrained(model_path) flax_model = FlaxAutoModelForSeq2SeqLM.from_pretrained(model, from_pt=True) ``` But not all...

I can't get phi-3 to work with -ngl flag. It seems that offloading any layers results in a crash in llama_decode_internal

You can choose GPU with GGML_VULKAN_DEVICE= environment variable. 0 is first GPU or CPU depending on setup. I tried it out on two machines, one with accelerated Nvidia drivers, one...

Same here, segmentation fault, but on an old Linux x86_64 elitebook laptop. In this case it was -mavx causing the error. This compiles, but it runs pretty slow without mavx:...