LLMLingua
LLMLingua copied to clipboard
microsoft
[Bug]: ZeroDivisionError: float division by zero upon using compress_json with `use_llmlingua2`
Describe the bug
I am trying to use compress_json and encountering ZeroDivisionError
Steps to reproduce
from llmlingua import PromptCompressor
llm_lingua = PromptCompressor(
model_name="microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank",
use_llmlingua2=True,
device_map="cpu",
)
json_data = {
"query": "This is a question for LLM 1",
"answer": "This is an answer from LLM 1"
}
json_config = {
"query": {
"rate": 1.0,
"compress": False,
"value_type": "str",
"pair_remove": False
},
"answer": {
"rate": 0.4,
"compress": True,
"value_type": "str",
"pair_remove": False
}
}
compressed_data = llm_lingua.compress_json(json_data, json_config, use_keyvalue_level_filter=False)
Note the error.
Expected Behavior
The error should not come.
Logs
ZeroDivisionError Traceback (most recent call last)
Cell In[21], [line 6](vscode-notebook-cell:?execution_count=21&line=6)
[1](vscode-notebook-cell:?execution_count=21&line=1) llm_lingua = PromptCompressor(
[2](vscode-notebook-cell:?execution_count=21&line=2) model_name="microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank",
[3](vscode-notebook-cell:?execution_count=21&line=3) use_llmlingua2=True,
[4](vscode-notebook-cell:?execution_count=21&line=4) device_map="cpu",
[5](vscode-notebook-cell:?execution_count=21&line=5) )
----> [6](vscode-notebook-cell:?execution_count=21&line=6) compressed_data = llm_lingua.compress_json(x[0], json_config, use_keyvalue_level_filter=False)
File ~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:242, in PromptCompressor.compress_json(self, json_data, json_config, instruction, question, rate, target_token, iterative_size, use_sentence_level_filter, use_keyvalue_level_filter, use_token_level_filter, keep_split, keep_first_sentence, keep_last_sentence, keep_sentence_number, high_priority_bonus, context_budget, token_budget_ratio, condition_in_question, reorder_keyvalue, condition_compare, rank_method)
[240](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:240) print(f">> {context}")
[241](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:241) print(f">> {force_context_ids}")
--> [242](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:242) compressed_res = self.structured_compress_prompt(
[243](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:243) context=context,
[244](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:244) instruction=instruction,
[245](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:245) question=question,
[246](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:246) rate=rate,
[247](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:247) target_token=target_token,
[248](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:248) iterative_size=iterative_size,
[249](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:249) force_context_ids=force_context_ids,
[250](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:250) use_sentence_level_filter=use_sentence_level_filter,
[251](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:251) use_context_level_filter=use_keyvalue_level_filter,
[252](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:252) use_token_level_filter=use_token_level_filter,
[253](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:253) keep_split=keep_split,
[254](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:254) keep_first_sentence=keep_first_sentence,
[255](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:255) keep_last_sentence=keep_last_sentence,
[256](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:256) keep_sentence_number=keep_sentence_number,
[257](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:257) high_priority_bonus=high_priority_bonus,
[258](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:258) context_budget=context_budget,
[259](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:259) token_budget_ratio=token_budget_ratio,
[260](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:260) condition_in_question=condition_in_question,
[261](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:261) reorder_context=reorder_keyvalue,
[262](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:262) condition_compare=condition_compare,
[263](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:263) add_instruction=False,
[264](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:264) rank_method=rank_method,
[265](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:265) concate_question=False,
[266](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:266) strict_preserve_uncompressed=False,
[267](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:267) )
[268](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:268) compressed_json_text = remove_consecutive_commas(
[269](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:269) compressed_res["compressed_prompt"]
[270](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:270) )
[271](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:271) compressed_res["compressed_prompt"] = json.loads(compressed_json_text)
File ~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:394, in PromptCompressor.structured_compress_prompt(self, context, instruction, question, rate, target_token, iterative_size, force_context_ids, force_context_number, use_sentence_level_filter, use_context_level_filter, use_token_level_filter, keep_split, keep_first_sentence, keep_last_sentence, keep_sentence_number, high_priority_bonus, context_budget, token_budget_ratio, condition_in_question, reorder_context, dynamic_context_compression_ratio, condition_compare, add_instruction, rank_method, concate_question, strict_preserve_uncompressed)
[387](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:387) rate = target_token / sum(context_tokens_length)
[388](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:388) (
[389](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:389) context,
[390](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:390) context_segs,
[391](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:391) context_segs_rate,
[392](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:392) context_segs_compress,
[393](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:393) ) = self.segment_structured_context(context, rate)
--> [394](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:394) return self.compress_prompt(
[395](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:395) context,
[396](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:396) instruction,
[397](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:397) question,
[398](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:398) rate,
[399](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:399) target_token,
[400](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:400) iterative_size,
[401](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:401) force_context_ids,
[402](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:402) force_context_number,
[403](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:403) use_sentence_level_filter,
[404](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:404) use_context_level_filter,
[405](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:405) use_token_level_filter,
[406](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:406) keep_split,
[407](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:407) keep_first_sentence,
[408](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:408) keep_last_sentence,
[409](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:409) keep_sentence_number,
[410](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:410) high_priority_bonus,
[411](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:411) context_budget,
[412](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:412) token_budget_ratio,
[413](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:413) condition_in_question,
[414](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:414) reorder_context,
[415](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:415) dynamic_context_compression_ratio,
[416](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:416) condition_compare,
[417](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:417) add_instruction,
[418](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:418) rank_method,
[419](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:419) concate_question,
[420](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:420) context_segs=context_segs,
[421](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:421) context_segs_rate=context_segs_rate,
[422](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:422) context_segs_compress=context_segs_compress,
[423](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:423) strict_preserve_uncompressed=strict_preserve_uncompressed,
[424](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:424) )
File ~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:537, in PromptCompressor.compress_prompt(self, context, instruction, question, rate, target_token, iterative_size, force_context_ids, force_context_number, use_sentence_level_filter, use_context_level_filter, use_token_level_filter, keep_split, keep_first_sentence, keep_last_sentence, keep_sentence_number, high_priority_bonus, context_budget, token_budget_ratio, condition_in_question, reorder_context, dynamic_context_compression_ratio, condition_compare, add_instruction, rank_method, concate_question, context_segs, context_segs_rate, context_segs_compress, target_context, context_level_rate, context_level_target_token, return_word_label, word_sep, label_sep, token_to_word, force_tokens, force_reserve_digit, drop_consecutive, chunk_end_tokens, strict_preserve_uncompressed)
[469](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:469) """
[470](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:470) Compresses the given context.
[471](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:471)
(...)
[534](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:534) - "saving" (str): Estimated savings in GPT-4 token usage.
[535](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:535) """
[536](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:536) if self.use_llmlingua2:
--> [537](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:537) return self.compress_prompt_llmlingua2(
[538](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:538) context,
[539](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:539) rate=rate,
[540](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:540) target_token=target_token,
[541](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:541) use_context_level_filter=use_context_level_filter,
[542](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:542) use_token_level_filter=use_token_level_filter,
[543](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:543) target_context=target_context,
[544](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:544) context_level_rate=context_level_rate,
[545](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:545) context_level_target_token=context_level_target_token,
[546](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:546) force_context_ids=force_context_ids,
[547](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:547) return_word_label=return_word_label,
[548](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:548) word_sep=word_sep,
[549](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:549) label_sep=label_sep,
[550](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:550) token_to_word=token_to_word,
[551](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:551) force_tokens=force_tokens,
[552](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:552) force_reserve_digit=force_reserve_digit,
[553](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:553) drop_consecutive=drop_consecutive,
[554](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:554) chunk_end_tokens=chunk_end_tokens,
[555](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:555) )
[556](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:556) assert (
[557](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:557) rate <= 1.0
[558](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:558) ), "Error: 'rate' must not exceed 1.0. The value of 'rate' indicates compression rate and must be within the range [0, 1]."
[560](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:560) if not context:
File ~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:928, in PromptCompressor.compress_prompt_llmlingua2(self, context, rate, target_token, use_context_level_filter, use_token_level_filter, target_context, context_level_rate, context_level_target_token, force_context_ids, return_word_label, word_sep, label_sep, token_to_word, force_tokens, force_reserve_digit, drop_consecutive, chunk_end_tokens)
[925](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:925) return res
[927](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:927) if target_token > 0:
--> [928](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:928) rate = min(target_token / n_original_token, 1.0)
[930](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:930) if use_token_level_filter:
[931](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:931) compressed_context, word_list, word_label_list = self.__compress(
[932](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:932) context_chunked,
[933](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:933) reduce_rate=max(0, 1 - rate),
(...)
[938](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:938) drop_consecutive=drop_consecutive,
[939](https://vscode-remote+ssh-002dremote-002b192-002e168-002e1-002e11.vscode-resource.vscode-cdn.net/home/pranav/workspace/merokudao/friday/notebooks/~/workspace/merokudao/friday/venv/lib/python3.12/site-packages/llmlingua/prompt_compressor.py:939) )
ZeroDivisionError: float division by zero
Additional Information
I patched the code locally to try to print and see if something went wrong, it seems like the tokens are empty strings
[{'query': 'This is a question for LLM 1', 'answer': 'This is an answer from LLM 1'}]
>>> Doing data {'query': 'This is a question for LLM 1', 'answer': 'This is an answer from LLM 1'}
>> ['<llmlingua, compress=False>{</llmlingua>', '<llmlingua, compress=False>"query": "This is a question for LLM 1", </llmlingua>', '<llmlingua, compress=False>"answer": "</llmlingua><llmlingua, rate=0.4>This is an answer from LLM 1</llmlingua><llmlingua, compress=False>"</llmlingua>', '<llmlingua, compress=False>}</llmlingua>']
>> [0, 1, 2, 3]
>> ['', '', '', '']
>> ['', '', '', '']
>> 0
>> ['', '', '', '']
>> 0
>> ['', '', '', '']
>> 0
>> ['', '', '', '']
>> 0
Hi @r4881t, thanks for your feedback. However, LLMLingua-2 currently does not support the compression_json mode, so there may be some bugs.
