LoopControl

I'm running into this issue now with increased traffic -- every request is getting timed out with messages like: ``` error copying response: readfrom tcp 127.0.0.1:8080->127.0.0.1:39936: write tcp 127.0.0.1:8080->127.0.0.1:39936: write:...

@User-26 did you find a solution for this? I'm noticing the same issue with cache not being used

+1 on this. As @Ph0rk0z mentioned, it'd be great to not have it auto-load the full-precision version every time I want to set 4bit mode and play with groupsize settings...

> The whole purpose of loading the model specific settings was to seamlessly switch between models The purpose of the model-settings (very convenient by the way) is so we don’t...

Just tested - works great thanks!

> Yup.. will just change my config file to keep not loading the model by default. Hadn’t realized it was customizable via config file too - thanks for the tip!

Could you briefly describe how to convert an OPT huggingface model to .pt (or provide a link to pregenerated .pt)? Would it be similar to this command documented in the...

Good news is, after quantizing a 13B Erebus `pt`, the model loads in around 8GB of VRAM and seems to generate text. Problem is, I'm seeing 5x+ slower generations with...

When I do `npm install 'git+https://github.com/bestguy/sveltestrap.git#pull/574/head'` to install from this pull request, I'm getting the following error: ``` npm ERR! Found: [email protected] npm ERR! node_modules/svelte npm ERR! dev svelte@"^4.0.5" from...

Still getting error after the 3 commits above. When I `--force` the install it works with this error: ``` npm WARN using --force Recommended protections disabled. npm WARN ERESOLVE overriding...