jmoney7823956789378

I could do the same with an MI25 or MI60, if that was something you wanted.

Personally I'm trying to get my personally-trained one up on exllama. Unfortunately I haven't even been getting any confirmation that it loads correctly.

I tried a small finetune using a separate machine holding an RTX 2080S. (I'm considering using runpod later) Utilized the monkey-patch with a small 7b 4bit model to finetune on...

Ah, I gotcha. I trained soley within the ooba ui, using a plaintext dataset of approximately 33K lines of mostly transcripts from audio lectures (about cyber forensics). As I am...

I'm also thankful for your efforts on this. Just retrained a 33B lora (had to rent compute since split gpu training was buggy) and it seems to be working somewhat....

For perplexity you can run test_benchmark_inference.py with the `-ppl` option. Pretty sure turboderp himself tested this scaling yesterday and found perplexity to massively increase in exchange for context length.

Unsure why you are using the rocm torch version when you are using an nvidia tesla T4, but try using the normal version.

No problem, not sure what the other guy meant with ninja... the fact that you're getting an error code and message from ninja means you have it installed at the...

The -d option should be used with the model's folder, rather than the model file. It also seems like you're missing some of the rocm hip libraries. Are you on...

Hey fellow MI60 chad. Exllama in ooba's webui recent changed to using the pip module. Try `python -m pip install git+https://github.com/jllllll/exllama`