Compatible models

[oz03-hub]

Hello I want to know what models can be used to compress using llmlingua, I know default is NousResearch/Llama-2-7b-hf, there is also a gpt-2 alternative etc. On hugging face how do you filter which models can be used? Or is there a list so that I can experiment?

Thanks

[iofu728]

Hi @oz23-hub, thank you for your interest in LLMLingua. We believe you can use any language model as a compressor in LLMLingua. The principle is that the stronger a small language model is in a specific domain, the better its performance usually is. For more details, you can refer to: https://github.com/microsoft/LLMLingua/discussions/57