AJ Walter

Ha! I'll see what I can do about that. Good find.

If anyone needs a known good hash to roll back to, you can reset here (make sure to run this in the GPTQ-for-LLaMa repo, of course) ``` git reset --hard...

> Did you get the model to output predictions in your container? Mine appears to load the model, but throws an error on prediction. Yes, it's working for me with...

> I 'fixed' inference by: \ That would make sense - you need to also rebuild the cuda package with the .cpp files from that commit. The container starts fresh...

Also causes a crash dump on actual hardware.

If anyone happens to want to use Podman instead, I have a repo here called [Text-Generation-Webui-Podman](https://github.com/RedTopper/Text-Generation-Webui-Podman). It compiles and installs the GPTQ-for-LLaMa repo so 4bit works too. In theory the...

@oobabooga If it helps your cause, I updated my repo to also support Docker. It has some extras goodies like persisting the container's data, having a smaller final build size,...