BBC-Esq
You can use something like this: ```model.client[0].auto_model = model.client[0].auto_model.to(torch_dtype)``` However, you'll need to import the following: ```from torch.cuda.amp import autocast, GradScaler``` You can use it after instantiating the model. Unfortunately,...
> It works just fine for me. For GPU, you need to seek other solutions (https://discuss.pytorch.org/t/does-dynamic-quantization-support-gpu/119231). Do you have a sample script or series of commands that you used? I've...
> I just used the code from the quantization section in the readme of this repo. Thanks. I can't get it to work. If I try to get the error...
I figured out how to dynamically quantize the instructor-xl model, but at the point that it's supposed to create the embeddings, i want it to use gpu acceleration (cuda) just...
> @BBC-Esq did you try to use the `load_dir_path` function from https://github.com/UKPLab/sentence-transformers/blob/66e0ee30843dd411c64f37f65447bb38c7bf857a/sentence_transformers/util.py#L552-L554 ? Because it's been there from version 2.3.0 and looks exactly like what is needed here. Couldn't figure...
> I made PR [BBC-Esq#1](https://github.com/BBC-Esq/instructor-embedding/pull/1) which mimicks the logic of `load_dir_path` while working well with instructor. I managed to make it run for both huggingface and local models. Is this...
Please take a few minutes to review this pull request and address @racinmat 's questions and modifications as well. If you can't find the time to do this, please transfer...
> Now I understand the problem, the files `https://huggingface.co/hkunlp/instructor-large/blob/main/config_sentence_transformers.json` etc. need to be downloaded and the https://github.com/UKPLab/sentence-transformers/blob/66e0ee30843dd411c64f37f65447bb38c7bf857a/sentence_transformers/util.py#L559 does not let you match files in the root directory. I heavily changed...
The owners of this repository don't really update anymore, but you can try my fork here instead, which is compatible with the newest version of ```sentence-transformers```: https://github.com/BBC-Esq/instructor-embedding
A fork of my fork! lol I like it. What else did you add?