[Question]: How to utilize compression to a finetuned LLM?

[LiuZhihhxx]

Describe the issue

I have an LLM finetuned for a down-stream task using input-output pairs data(X_train - Y_train). Now I plan to utilize llmlingua2 to compress X_test --> X_test_compress and evaluate the performance improvement. Is it necessary to re-fintune the LLM using a X_train_compress ? If so, it seems like to finetune the LLM to fit the compressor maybe?😂

Any advice is appreciated!

[iofu728]

Hi @LiuZhihhxx, thanks for your interest in LLMLingua.

Based on your description, you have fine-tuned an LLM for a specific task. I recommend following the instructions at this link to construct the compressed dataset for these data and then fine-tune the LLMLingua-2 compressor.