[BUG] rerank 报错 IndexError: list index out of range

[lordk911]

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

• [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

• [X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

1、develop_for_v1.3.1 纯python 环境运行 2、运行脚本bash scripts/run_for_openai_api_with_cpu_in_Linux_or_WSL.sh

cat scripts/run_for_openai_api_with_cpu_in_Linux_or_WSL.sh
      #!/bin/bash
      bash scripts/base_run.sh -s "LinuxOrWSL" -w 8 -m 19530 -q 8777 -c -o -b 'http://10.9.123.456:9997/v1' -k 'sk-abc' -n 'qwen1.5-14B-Chat' -l '4096'

3、修改了代码：qanything_kernel/connector/llm/llm_for_openai_api.py

4、在做知识问答时，有时会遇到错误：

Traceback (most recent call last):
  File "handle_request", line 132, in handle_request
    "_asgi_lifespan",
  File "/data/miniconda3/envs/qanything-python/lib/python3.10/site-packages/sanic/response/types.py", line 547, in stream
    await self.streaming_fn(self)
  File "/data/llm-project/QAnything/qanything_kernel/qanything_server/handler.py", line 351, in generate_answer
    async for resp, next_history in local_doc_qa.get_knowledge_based_answer(
  File "/data/llm-project/QAnything/qanything_kernel/core/local_doc_qa.py", line 199, in get_knowledge_based_answer
    retrieval_documents = self.rerank_documents(query, retrieval_documents)[: self.rerank_top_k]
  File "/data/llm-project/QAnything/qanything_kernel/core/local_doc_qa.py", line 181, in rerank_documents
    scores = self.local_rerank_backend.predict(query, [doc.page_content for doc in source_documents])
  File "/data/llm-project/QAnything/qanything_kernel/connector/rerank/rerank_backend.py", line 87, in predict
    tot_batches, merge_inputs_idxs_sort = self.tokenize_preproc(query, passages)
  File "/data/llm-project/QAnything/qanything_kernel/connector/rerank/rerank_backend.py", line 67, in tokenize_preproc
    qp_merge_inputs = self.merge_inputs(query_inputs, passage_inputs)
  File "/data/llm-project/QAnything/qanything_kernel/connector/rerank/rerank_backend.py", line 43, in merge_inputs
    chunk1['attention_mask'].append(chunk2['attention_mask'][0])
IndexError: list index out of range

期望行为 | Expected Behavior

1、目前调用openai_api的方式感觉比较慢 2、不出现遇到的错误

运行环境 | Environment

- OS:Ubuntu 20.04 
- NVIDIA Driver:550.54.14
- CUDA: 12.4
- docker:
- docker-compose:
- NVIDIA GPU: A40
- NVIDIA GPU Memory: 48G

QAnything日志 | QAnything logs

No response

复现方法 | Steps To Reproduce

No response

备注 | Anything else?

No response

[sun-888]

我也遇到了这个错误，不知道你解决了没有

[cakat]

遇到同样的问题，来找答案

[ja3592]

猫腻出在base_run.sh文件的 openai_api_context_length="4096",这里限制了文本长度，要把这个数值扩大一些

[sun-888]

猫腻出在base_run.sh文件的 openai_api_context_length="4096",这里限制了文本长度，要把这个数值扩大一些

报错出在rerank阶段，你说的这个应该是之后api请求时才会用到的，应该没关联吧

[exlazybobby]

猫腻出在base_run.sh文件的 openai_api_context_length="4096",这里限制了文本长度，要把这个数值扩大一些

有效

[knightwj]

猫腻出在base_run.sh文件的 openai_api_context_length="4096",这里限制了文本长度，要把这个数值扩大一些

有效

遇到同样的问题，改大数值没有解决问题