linuxmagic-mp

max_memory=max_memory, NameError: name 'max_memory' is not defined For testing, running on a smaller machine, 128GB, 48GB GPU ram Maybe the example could use a suggestion eg... max_memory={0: "44GiB", "cpu": "110GiB"})

Minor first run errors.. Trying to use this with Falcon-40b, just passing along error outputs to the console at this point.. ~/git/LLM-As-Chatbot/app.py:1030: UserWarning: You have unused kwarg parameters in Row,...

Doesn't see Falcon 40b converted, to be a GGML file..ctransformers problem?

4

On Ubuntu Focal.. ``` Current binding: c_transformers Current model: None Menu: 1 - WizardLM-Uncensored-Falcon-40b.reference 2 - Install model 3 - Change binding 4 - Back Enter your choice: 1 You...

Minor: Readme or installation might suggest installing the correct tk package in Ubuntu

1

Not everyone wants to run the installer (webui.sh), and instead will just try the app.py, and the 'pip' requirements, but on Ubuntu you will also have to install the proper...

May not even be a Transformers issue.. WizardLM-Uncensored-Falcon-40

1

Just could use some feedback on debugging with ctransformers, have a strange case where things are generally working, but occasionally I don't get output... using /models/WizardLM-Uncensored-Falcon-40b/ggml-model-falcon-40b-wizardlm-qt_k5.bin (GGML) ``` tokens =...

Could we also get the Makefile updated to build a libfalcon.so

3

Still would be nice when everything starts getting merged into llama, for one place to maintain.. I just don't like touch the C files and embarassing my self, but should...

Problem with cMake on Linux focal, Cuda

12

# Prerequisites Following the instructions in the README, Linux Ubuntu Focal rm -rf build; mkdir build; cd buil cmake -DLLAMA_CUBLAS=1 .. cmake --build . --config Release # Expected Behavior Simply...