Errors may occur when calculating timestamps：“IndexError: list index out of range”

[xubzhlin]

trafficstars

🐛 Bug

Errors may occur when calculating timestamps：“IndexError: list index out of range”

Code sample

imput is BytesIO

model = AutoModel(
            model=model,
            vad_model=vad_model,
            spk_model=spk_model,
            spk_mode="vad_segment",
            device=device,
            disable_update=True
        )
model.generate(
            input=input,
            cache=cache,
            language=language,
            use_itn=use_itn,
            batch_size_s=batch_size_s,
            merge_vad=False,
            output_timestamp=True
        )

  File "E:\ai\asr\venv\lib\site-packages\funasr\auto\auto_model.py", line 306, in generate
    return self.inference_with_vad(input, input_len=input_len, **cfg)
  File "E:\ai\asr\venv\lib\site-packages\funasr\auto\auto_model.py", line 464, in inference_with_vad
    results = self.inference(
  File "E:\ai\asr\venv\lib\site-packages\funasr\auto\auto_model.py", line 345, in inference
    res = model.inference(**batch, **kwargs)
  File "E:\ai\asr\venv\lib\site-packages\funasr\models\sense_voice\model.py", line 943, in inference
    timestamp.append([tokens[token_id], ts_left, ts_right])
IndexError: list index out of range

Environment

• OS :Windwos
• FunASR Version :1.2.6
• ModelScope Version :1.27.0
• PyTorch Version :2.4.1
• How you installed funasr :pip
• Python version:3.8.8

Fix

• funasr/models/sense_voice（第 936-943 行）
• Original

                for pred_token, pred_frame in pred:
                    _end = _start + len(list(pred_frame))
                    if pred_token != 0:
                        ts_left = max((_start * 60 - 30) / 1000, 0)
                        ts_right = min((_end * 60 - 30) / 1000, (ts_max * 60 - 30) / 1000)
                        timestamp.append([tokens[token_id], ts_left, ts_right])
                        token_id += 1
                    _start = _end

• Fixed

                for pred_token, pred_frame in pred:
                    _end = _start + len(list(pred_frame))
                    if pred_token != 0:
                        ts_left = max((_start * 60 - 30) / 1000, 0)
                        ts_right = min((_end * 60 - 30) / 1000, (ts_max * 60 - 30) / 1000)
                        # 这里做一个判断
                        if(len(tokens) <= token_id):
                            break
                        timestamp.append([tokens[token_id], ts_left, ts_right])
                        token_id += 1
                    _start = _end