ItsLogic

It fixes and breaks a few things. Two non hostile mobs are now at the right height while two are now sideways instead of upright  The wither skeleton...

> should consider getting rid of stale bot? or make it less strict? I dont think its a bad idea but it probably shouldnt auto close issues. Getting a stale...

> 2-bit LLaMA!? What's the loss in quality at that point? That's phenomenal if it's not significant! From the benchmarks provided in the GPTQ repo loss in quality for 2bit...

> I think that removing `--load-in-4bit` would lead to confusion now that it is already implemented and being used. Maybe let `--load-in-4bit` and `--llama-bits` coexist? Yeah I can see it...

Both `--load-in-4bit` and `--llama-bits` now coexist. I changed the description of `--llama-bits` to better describe the functionality (including the support for pre-quantized 8bit) but ill leave the name until you...

> Did you get any results there? In terms of performance/ hardware usage. I actually just made a pull request to the GPTQ repo with my results for 3 and...

I must say this running on my phone at all was surprising. Here are my results on an 8+gen1 for 4bit 7B  and results for my desktop with 13900k...

I do plan to create an easily accessible list in the future but it will take quite a bit of time so for now you will need to grab it...

I will record a short video today

The video is currently uploading. > Let me know if you want me to send you these via a different platform or just complete the full list that I have....