[Question]: Is It Possible to Share the Code how to Do the evaluation

[ZhichaoWang970201]

Describe the issue

I am wondering if you can share the evaluation code for "LongLLMLingua mitigates the 'lost in the middle' issue in LLMs, enhancing long-context information processing."

[iofu728]

Hi @zhichaoWang970201, thanks for your support.

You can follow the instructions in longchat-13b-16k to run the NaturalQA benchmark.

[ZhichaoWang970201]

Thank you for your sharing.

In addition, the data used for generating answer is already very short. Is this what the author used for prompt compressing? If not, will the author add prompt to ask LLM to copy exact text from the input prompt? The evaluation file asks for exactly matching.

Thank you.

[ZhichaoWang970201]

The problem is solved when reading more of the lost in the middle paper.