RUC-NLPIR
运行simple_pipeline.py 出错
环境
pip install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu121 torch2.5.1依旧报错 vllm 0.5.5 faiss-cpu使用是的1.8.0 flashrag 使用的是main分支
报错内容
output_dataset = pipeline.run(test_data, do_eval=True)
TypeError Traceback (most recent call last) Cell In[9], line 1 ----> 1 output_dataset = pipeline.run(test_data, do_eval=True)
File /media/cheng/code/00-learn_code/learn_python/flashrag/FlashRAG-main/flashrag/pipeline/pipeline.py:125, in SequentialPipeline.run(self, dataset, do_eval, pred_process_fun) 123 if self.refiner: 124 del self.refiner --> 125 pred_answer_list = self.generator.generate(input_prompts) 126 dataset.update_output("pred", pred_answer_list) 128 dataset = self.evaluate(dataset, do_eval=do_eval, pred_process_fun=pred_process_fun)
File /media/cheng/code/00-learn_code/learn_python/flashrag/FlashRAG-main/flashrag/generator/generator.py:408, in HFCausalLMGenerator.generate(self, input_list, batch_size, return_scores, return_dict, **params) 400 batched_prompts = input_list[idx : idx + batch_size] 401 inputs = self.tokenizer( 402 batched_prompts, 403 return_tensors="pt", (...) 406 max_length=self.max_input_len, 407 ).to(self.model.device) --> 408 outputs = self.model.generate( 409 **inputs, 410 output_scores=True, 411 return_dict_in_generate=True, 412 **generation_params, 413 ) 415 generated_ids = outputs.sequences 416 logits = torch.stack(outputs.scores, dim=1).softmax(-1)
File ~/miniconda3/envs/env_flashrag/lib/python3.11/site-packages/torch/utils/_contextlib.py:116, in context_decorator.
File ~/miniconda3/envs/env_flashrag/lib/python3.11/site-packages/transformers/generation/utils.py:2004, in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, **kwargs)
2001 batch_size = inputs_tensor.shape[0]
2003 device = inputs_tensor.device
-> 2004 self._prepare_special_tokens(generation_config, kwargs_has_attention_mask, device=device)
2006 # decoder-only models must use left-padding for batched generation.
2007 if not self.config.is_encoder_decoder and not is_torchdynamo_compiling():
2008 # If input_ids was given, check if the last id in any sequence is pad_token_id
2009 # Note: If using, inputs_embeds this check does not work, because we want to be more hands-off.
File ~/miniconda3/envs/env_flashrag/lib/python3.11/site-packages/transformers/generation/utils.py:1820, in GenerationMixin._prepare_special_tokens(self, generation_config, kwargs_has_attention_mask, device) 1817 return torch.tensor(token, device=device, dtype=torch.long) 1819 bos_token_tensor = _tensor_or_none(generation_config.bos_token_id, device=device) -> 1820 eos_token_tensor = _tensor_or_none(generation_config.eos_token_id, device=device) 1821 pad_token_tensor = _tensor_or_none(generation_config.pad_token_id, device=device) 1822 decoder_start_token_tensor = _tensor_or_none(generation_config.decoder_start_token_id, device=device)
File ~/miniconda3/envs/env_flashrag/lib/python3.11/site-packages/transformers/generation/utils.py:1817, in GenerationMixin._prepare_special_tokens.
TypeError: 'NoneType' object cannot be interpreted as an integer
在generator.py的400行打印一下batched_prompts以及tokenize后的inputs看看有没有问题。或者把framework换成vllm试试。
