Huiqiang Jiang

Hi @gayuoptisol, You can set the `device_map` to “cpu” as follow, ```python llm_lingua = PromptCompressor(device_map="cpu") ```

Hi @jwahnn, Thank you for your support of LLMLingua. You can directly use the current code to compress prompts and input them into LLaMA. Experiments in LongLLMLingua have shown that...

Hi @jwahnn, The link at https://github.com/microsoft/LLMLingua#2-using-longllmlingua-for-prompt-compression is just a quick start guide on how to use our library. For more detailed information and examples, please refer to our [documentation](https://github.com/microsoft/LLMLingua/blob/main/DOCUMENT.md) and...

Hi @izhx, can you add a LICENSE file in the repo clarifying the licenses for the datasets used in the paper? This is essential for other to properly leverage what...

+1.

Hi @zhichaoWang970201, thanks for your support. You can follow the instructions in [longchat-13b-16k](https://github.com/nelson-liu/lost-in-the-middle/blob/main/EXPERIMENTS.md#longchat-13b-16k) to run the NaturalQA benchmark.

Hi @yyjabiding, thanks for your interest in LLMLingua. Although we haven't tested it, it seems possible. LLMLingua-2 forwards a BERT-level model chunk by chunk, so increasing the batch size could...

Hi @xvyaward, thanks for your support in LLMLingua-2 and share detailed results. These results seem quite good and are generally similar to ours. I would like to confirm which specific...

Hi @LiuZhihhxx, thanks for your interest in LLMLingua. Based on your description, you have fine-tuned an LLM for a specific task. I recommend following the instructions at [this link](https://github.com/microsoft/LLMLingua/tree/main/experiments/llmlingua2/data_collection) to...

Hi @AlexHe99 and @JOY-SWang, thanks for your question. what’s your local `transformers` version? Try to upgrade it **>4.36.0**.