The Objective Dad

+1 for this feature request. My use case requires high-quality embeddings for agent memories that are periodically generated. Very similar to section 4.2 from [this paper](https://dl.acm.org/doi/pdf/10.1145/3586183.3606763). TEI is a good...

+1 AutoGPTQ version: 0.6.0 Pretrained model is a merged qlora based on dolphin-2.5-mixtral-8x7b This is the code I am using to quantize: ```python gptq_config = GPTQConfig( bits=4, dataset=["/data/open-instruct"], tokenizer=tokenizer, group_size=-1,...

@eraclea2 This is the [workaround](https://gist.github.com/theobjectivedad/11440f1aa65ee6e26cb1e23ae3424a3d) I'm using. Use at your own risk ... also I'm not 100% sure if this is getting triggered from another underlying problem. I can say...

+1

+1

If anyone is looking for a hack, this is what I am using until this issues is resolved. The ``key_exists`` function below will check for true existence of ``key``: ```python...

Downgrading to pyarrow 11 worked for me and allows me to use ``paged_adamw_32bit``. The issue happens whenever I resume training. I don't see a downside to the downgrade however it...

+1

For folks stuck on this, here is a quick hack you can use ... basically you can just map ``/etc/ssl/certs/ca-certificates.crt`` in the container to a another file that contains your...