LLMLingua icon indicating copy to clipboard operation
LLMLingua copied to clipboard
verified publisher icon microsoft

[Question]: longbench数据集测试对context和question的压缩

Open Diana303068 opened this issue 1 year ago • 2 comments

Describe the issue

longbench数据集中有的数据集的问题很长，仅把context压缩到2000token会超过4096个token的限制，请问这块是怎么处理的？

Diana303068 avatar Jun 05 '24 10:06 Diana303068