Forkoz

I found out that compiling the C extension for multiple architectures causes import errors.

Source of the error is: ``` def _modify_model( m, enable_cnn_optimization=True, enable_fused_linear_geglu=True, prefer_lowp_gemm=True, enable_triton=False, enable_triton_reshape=False, enable_triton_layer_norm=False, memory_format=None, ): ``` inside: **https://github.com/chengzeyi/stable-fast/blob/main/src/sfast/compilers/diffusion_pipeline_compiler.py** ` enable_fused_linear_geglu=True,` Geglu always enables even on unsupported architecture. I...

For me too.. I don't have a bing API key nor can I really get one.

U got the wrong llama.cpp probably

``` # llama-cpp-python without GPU support llama-cpp-python==0.1.83; platform_system != "Windows" https://github.com/abetlen/llama-cpp-python/releases/download/v0.1.83/llama_cpp_python-0.1.83-cp310-cp310-win_amd64.whl; platform_system == "Windows" # llama-cpp-python with CUDA support https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.1.83+cu117-cp310-cp310-win_amd64.whl; platform_system == "Windows" https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.1.83+cu117-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and platform_machine ==...

Uninstall the GGML one.

Hmm.. I think you have to uninstall the actual packages inside the VENV and install only the GGUF one. I also can't use GGML + GGUF installed at the same...

It's supposed to work now. Just go into the venv as above and do pip install https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.1.83+cu117-cp310-cp310-win_amd64.whl

>mbination: 00000000, 00000000; is this really a GGML file? The model is fine, it's clearly loading with the old version and expecting GGML. For me too, I cannot use GGUF...

It didn't work for me even though I renamed both versions to be the same as the wheels. I have both ggml_cuda and a cuda but it picks the wrong...