0xbitches

> yes, but once I get my a100s fixed because there definetly is something throttling them Thanks - I just finetuned my own lora with tloen's code, unfortunately the results...

> inference kwargs from this repo Could you specify what you meant here? Did you use the alpaca code and train your own model?

Getting this error with `device_map` set to auto and `load_in_8bit` set to true. Any possible cause?

Using the fix in #31, however, the results from 4 bit models are still repetitive nonsense. FP16 works but the results are also very bad. Relevant spec: Intel-13700k, 240ms/token Built...

For now a simple button that saves the current output in notebook/chat mode to a .txt file should suffice. I think the equivalent of automatic1111's implementation of "save image" function...

Also, would be great if we can have 4bit support by incorporating GPTQ #2

> With 256 tokens the loss slowly pulls further down to somewhere slightly above 0.8. You could maybe get away with using 2 epochs instead of 3, though. Yeah I...

@deep-diver which model is this playground running? also what hardware?

@devilismyfriend 0.37.0 latest release

Changing device_map to cpu did not help for me, still getting the same stack trace. It looks like the downloaded model is using `{'base_model': 0}` device map, which only loads...