0xbitches

Limit CPU RAM usage when offloading model to GPU

5

**Description** Would like to request a feature to limit python's RAM usage when loading a model to GPU. Right now, loading llama-30b-int4 would use up all of my 32GB of...

Save logs to a local folder in normal and --notebook mode

3

As titled. As this project is gaining attention it is quite necessary to set up a proper issue template, as well as document important keyboard shortcuts such as "ctrl+enter" in...

Benchmarking and optimization tips

5

Not exactly an issue, but have just been trying to run one epoch of finetuning with llama-13b. On a 4090 looks like it will take roughly 4 hours with the...

Finetuned Model Inference error: AttributeError: 'NoneType' object has no attribute 'device'

11

Update: for anyone experiencing this issue, see the workaround I posted in https://github.com/tloen/alpaca-lora/issues/14#issuecomment-1471263165 I tried out the finetune script locally and and it looks like there was no problem with...

Error when loading custom finetuned lora: AttributeError: 'NoneType' object has no attribute 'device'

3

### Describe the bug This is the same bug as https://github.com/tloen/alpaca-lora/issues/14. Here's the fix I used to workaround this issue: https://github.com/tloen/alpaca-lora/issues/14#issuecomment-1471263165 See suggested fix below. ### Is there an existing...

Cannot Reopen After Quitting on Mac

1

### Describe the bug Running on a Macbook pro with M2. Tried both app store and dmg installation. When I checked the "Quit: Minimize to Tray" option and tries quitting...

[Enhancement]: Cost Estimation & API Usage Telemetry

1

### Version VisualStudio Code extension ### Suggestion Hello, thanks for building the extension. Since it uses GPT-4 and could be potentially expensive, would it be possible to implement some form...