sharp-trickster

prompts queue but do not execute

5

Tried the "api example": I can read multiple "got prompt" on cmd but no execution at all. I open up the browser interface, and hit "queue prompt" there to test,...

Wont load quantized models

8

### Describe the bug Downloaded OPT, Galactica and even CodeGEN (not listed on Downloads, from huggingface), and they work fine. Tried the same with 4 bit quantized models (vicuna-13b-GPTQ-4bit-128g and...

Slow inference and low GPU use.

3

The inference.py and its running at ~4.2it/s. It barely loads my RTX2060 (0-13% use) The inference_speed_test script gives me ~33.2it/s on the same model and video settings. Changing the --workers...

`x, sr = li.load(f, sampling_rate)` throws "TypeError: load() takes 1 positional argument but 2 were given" I fixed mine by explicitly declaring the variable, [as the example in the librosa...