maxin9966

max_input_tokens随着文档长度增加？

3

限定死max_input_tokens为16K，现在只要文档长度超出一定范围，就报错： 2024-06-11 02:12:34,741 - parallel_doc_qa.py - 187 - INFO - Parallel Member Num: 142 2024-06-11 02:13:36,918 - parallel_doc_qa.py - 197 - INFO - Finished parallel_exec. Time spent: 62.17440700531006 seconds. 2024-06-11...

长文Agent ParallelDocQA每次对话都需要1分钟进行parallel_exec

2

## 使用了官方的长文agent例子，每次对话输入一个长文档，每次都需要花1分钟时间进行parallel_exec，请问这个60秒可以通过某些方法省略掉吗？： 2024-06-11 02:36:20,808 - parallel_doc_qa.py - 187 - INFO - Parallel Member Num: 142 2024-06-11 02:37:21,754 - parallel_doc_qa.py - 197 - INFO - Finished parallel_exec. Time spent: 60.94510245323181 seconds....

### System Info Name: torch Version: 2.4.0+cu124 OR Version: 2.3.1 Name: transformers Version: 4.44.2 CUDA Version: 12.5 ## Error: Traceback (most recent call last): File "/home/ma/work/flux/demo/TotoroUI/flux-demo-hyper.py", line 552, in gen...

### Expected Behavior 1 ### Actual Behavior 1 ### Steps to Reproduce hyper_lora = "Hyper-FLUX.1-dev-8steps-lora.safetensors" unet_lora = LoraLoaderModelOnly.load_lora_model_only(unet, hyper_lora, 0.125) cond, pooled = clip.encode_from_tokens(clip.tokenize(positive_prompt), return_pooled=True) cond = [[cond, {"pooled_output": pooled}]]...

长文档总结方案？

1

去年年中有提过，你们回复当前的方案并不能很好的处理长文档总结，后续会有针对长文档的专门agent，请问目前针对这个需求有好的方案吗？ 比如长文档速览，文档中的要点提取，书本推荐这类常见需求