sandeep-krutrim

Cannot train new tokens while finetuning

2

I am trying to add new tokens to the tokenizer and want to train them during finetuning. For that model embedding size has to be increased. For standard bert architectures...

Can you share details on fine-tuning for IndicGlue tasks like hyperparameters , full fine tuning or freezing the bert layers etc. Will be helpful in comparing fairly.

I am trying to reproduce the retrieval task results for Flores dataset of IndicXtreme. I am using. - fine-tuning/retrieval/retrieval.py code. However, the accuracy returned by the script is nowhere closer...

Why the numbers for XLMR are so weird in this table -

I am trying to speed up inference using quantized version of the llm2vec models. I have trained a gemma-2B-model on custom data. This is my inference code - ``` import...

Unable to load merged model for MTEB evaluation

2

I have trained a model using supervised contrastive. I saved the model using - `l2v.save('/llm2vec_models/final_merged_model', merge_before_save=True, save_config=True)` Now when I try to run mteb_eval.py using - `!python experiments/mteb_eval.py --model_name model_name...

Finetuning code for sequence classification, NLI task ?

1

Please provide. a script for finetuning the supervised contrastive models for sentence classification and NLI task. Is Lora Finetuning required or AutoModelforSequenceClassification can be used to train ?

Hi Can this be finetuned with LoRA without any additional script. Also, during finetuning, if we take sequence length of 512 or 1k, will it affect the inference for higher...

Hi, I am trying to train Llama 3.2 models using LLM2VEC. I am getting the following error - `ValueError: `rope_scaling` must be a dictionary with two fields, `type` and `factor`,...