Mastoca Space

I was able to go into the `repositories/GPTQ-for-LLaMa` directory and run `python setup_cuda.py` PS: you must be in your pyenv etc already before doing the setup above...etc.

I like the reloading idea as I've been switching to another model then returning to the updated model. I'm not sure what the app state would be in an 'unloaded'...

`python server.py --rwkv-cuda-on --rwkv-strategy "cuda fp16i8" --disk --gpu-memory 18` worked for me.

> > `--rwkv-cuda-on --rwkv-strategy "cuda fp16i8"` > > Those flags are for RWKV. OpenAssistant is something else ah, I see the error now. Thanks for clarifying the flags for me,...

@oobabooga I added a `print(reply)` in decode(output_ids) inside the `modules/text_generation.py` file and this was the output showing this.... ```Write a story about future of AI development In Write a story...

you may need to also add `max_memory={0: 18}` to `LlamaForCausalLM.from_pretrained` if u run into OOM errors when fine-tuning the 13B model

@222Phoenix how did your testing go? I've seen the same issue with a Kensington trackball on MacOS where scroll will jump after a certain distance.

I too turned Hide Completely on as soon as I found it. At first I thought the extension was broken or had css issues. I didn't uninstall right away as...

When you shutdown/reboot the host, (without docker compose down) the uwsgi instances sometimes seem to not get the signal to terminate from the bash script. This will cause a ~60...

Perhaps just hiding the layer-shell is ok as I doubt there's much need for an 'overview' of that layer. I could be wrong but as of now I don't even...