IB

> Would you be interested in making a PR for this? Seems like a useful edge case to close. I did make a fork but I can't seem to find...

Thanks for the response. > ExLlama pre-allocates the whole context, so it uses the same amount of VRAM (roughly) no matter how long your context is. Setting the max sequence...

Same error with -l 100 ``` (exllama) root@bb9b8f1170dd:/workspace/exllama# python example_chatbot.py -l 100 -ld ../text-generation-webui/loras/checkpoint-90/ -d ../text-generation-webui/models/TheBloke_Llama-2-70B-GPTQ/ -- Sequence length: 100 -- Temperature: 0.95 -- Top-K: 20 -- Top-P: 0.65 --...

> Hmm, I remember doing some napkin math when someone asked if 70B would fit in 40GB, and my estimate was that it would probably _just_ squeeze into 40GB (single-card)...

@EyeDeck > Is that an A100 40GB or 80GB? I think you can probably safely rule out OOMs if it's 80GB. Oh yeah sorry it's 80, I was on 48...

> hi，May I ask a slightly off-topic question? > > Did anyone who have compared alpaca_lora_4bit/autogptq and qlora? > > I remember is the qlorais half as fast as the...

This broke quite a few libraries, any fix?

Manually change the way your relationships are specified in your metadata to this format @iamolegga "using": { "manual_configuration": { "column_mapping": { "company_id": "company_id", "location_id": "id" }, "insertion_order": null, "remote_table": {...

I have a similar issue, it's very intermittent. Been working fine all week, but just had it happen 5 times in a row. To reproduce: - Boot laptop with external...

Using the command dm-tool lock **does** seem to work properly.