e5-mistral-7b-instruct
e5-mistral-7b-instruct copied to clipboard
kamalkraj
Finetune mistral-7b-instruct for sentence embeddings
I used this code and trained with Korean ko-snil data. adapter_config.json, adapter_model.safetensors, special_tokens_map.json, tokenizer_config.json, tokenizer.json, tokenizer.model 5 files were saved. I configured accelerate as shown below. I applied rola.json as...
Source code in `Accelerate` lib shows that `weights` in hooks is empty if the training task is launched via Deepspeed. https://github.com/huggingface/accelerate/blob/b8c85839531ded28efb77c32e0ad85af2062b27a/src/accelerate/accelerator.py#L2778-L2824 Threrfore, IndexError will be raised in `save_model_hook`. https://github.com/kamalkraj/e5-mistral-7b-instruct/blob/99021919b3c82bc67a4a897e8e9f39efe3d72cdc/peft_lora_embedding_semantic_search.py#L158-L162 -----------...
Hi, i do not understand why the execution fails when I use 2 GPUs, however with a single one it works. I tried the following setups: 1. Deafult configuration (i...
I got OOM during inference on 80G GPU with only input seqence length 1000. I tried to make a loop to feed input with 500 long, then I found GPU...
How can I solve the problem of OverFlow during the training steps? Thank you. The log is as follows. . [ [log.txt](https://github.com/user-attachments/files/15573074/log.txt) ](url)
Hello, for my research I’m planning to fine-tune the model using the HoVer dataset, which includes queries that can involve up to 4 documents for verification. I have a question...
