model support

[OliverLeeXZ]

Thanks for your great work! Can dola support more LLM model? Such as llama3.1, llama2, qwen2 or Mistral serious?

[voidism]

Hi @OliverLeeXZ

DoLa was supported by Huggingface transformers last year: https://huggingface.co/docs/transformers/main/generation_strategies#dola-decoding

However, we didn't test the decoding results on the latest models like Qwen, Llama3, etc. For reproducing the official results, please still follow the implementation in this repo. Thanks!