Huiqiang Jiang
Huiqiang Jiang
Hi @XiaoFengbing, sorry for the late reply. 2. Yes, we stated in the paper that it's 0.25. However, for tasks like MultiDocument QA, a setting of 0.3 or 0.4 tends...
Hi @junepark1, apologies for the late response, 1. Yes, you need to disable the reranker for now. We will update the results with the reranker enabled in the future. 2....
Hi @ulan-yisaev, thank you for your support and information. We will soon support the use of the Azure API for LLMLingua. Afterwards, we can extend the corresponding endpoint to other...
Just checked the script with @Twillghtaaa and found the main issue lies in the call mode of LLMs, with parameters largely consistent with those mentioned earlier. Experiments in LLMLingua and...
Hi @yfpeng1234, you can follow the https://github.com/microsoft/LLMLingua/blob/main/examples/RAG.ipynb and use "GPT-3.5-turbo-instruction" model.
Hi @danny-su, this design is intended for Linux machines. We plan to remove it soon.
Hi @danny-su, thank you for your suggestion. We will add a custom `torch_dtype`.
Hi @CEfanmin, thanks for your feedback. Since LLMLingua uses token-level prompt compression, it can indeed cause garbled text in some languages. You can try using LLMLingua-2.
Hi @oz23-hub, thank you for your interest in LLMLingua. We believe you can use any language model as a compressor in LLMLingua. The principle is that the stronger a small...
Hi @dingjingzhen, thanks for supporting LLMLingua. Could you provide more details about how you are using it and your environment? The LLMLingua series relies on a smaller model, such as...