"The MNTP LoRA weights are merged into the base model, and the trainable LoRA weights are initialized with SimCSE weights."

[cultivater]

"The MNTP LoRA weights are merged into the base model, and the trainable LoRA weights are initialized with SimCSE weights."

---

Hi, I saw this in your article, but I didn't find any corresponding configuration in your code. In your supervised contrastive learning (train_configs/supervised/MetaLlama3.json), there are only: "model_name_or_path": "meta-llama/Meta-Llama-3-8B-Instruct", "peft_model_name_or_path": "McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp"

may I know where is the checkpoint loading of SimCSE weights?