after training, the model with mmlu performace would downgrade?

[trouble-maker007]

trafficstars

[vaibhavad]

As the models are trained to be encoders and not be generative, it is very likely the MMLU performance of LLM2Vec models will be lower than their causal LM counterparts.

[vaibhavad]

Feel free to re-open if you have any more questions about this issue.

[trouble-maker007]

@vaibhavad thanks for the response, I just wonder after llm2vec, llm model a better text encoder than t5 in sd training