TinyChart推理报错TinyChartPhiForCausalLM.forward() got an unexpected keyword argument 'cache_position'

[chaserRen]

按照提供的inference脚本推理

我的脚本代码 `import torch from PIL import Image from tinychart.model.builder import load_pretrained_model from tinychart.mm_utils import get_model_name_from_path from tinychart.eval.run_tiny_chart import inference_model from tinychart.eval.eval_metric import parse_model_output, evaluate_cmds model_path = "/mnt/shared/TinyChart-3B-768"

tokenizer, model, image_processor, context_len = load_pretrained_model( model_path, model_base=None, model_name=get_model_name_from_path(model_path), device="cpu" # device="cpu" if running on cpu ) img_path = "docstructbench_enbook-zlib-o.O-17208435.pdf_105.jpg" prompt = r'''You are an AI assistant specialized in converting PDF images to Markdown format. Please follow these instructions for the conversion:

    1. Text Processing:
    - Accurately recognize all text content in the PDF image without guessing or inferring.
    - Convert the recognized text into Markdown format.
    - Maintain the original document structure, including headings, paragraphs, lists, etc.

    2. Mathematical Formula Processing:
    - Convert all mathematical formulas to LaTeX format.
    - Enclose inline formulas with \( \). For example: This is an inline formula \( E = mc^2 \)
    - Enclose block formulas with \\[ \\]. For example: \[ \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} \]

    3. Table Processing:
    - Convert tables to HTML format.
    - Wrap the entire table with <table> and </table>.

    4. Figure Handling:
    - Ignore figures content in the PDF image. Do not attempt to describe or convert images.

    5. Output Format:
    - Ensure the output Markdown document has a clear structure with appropriate line breaks between elements.
    - For complex layouts, try to maintain the original document's structure and format as closely as possible.

    Please strictly follow these guidelines to ensure accuracy and consistency in the conversion. Your task is to accurately convert the content of the PDF image into Markdown format without adding any extra explanations or comments.

   '''

response = inference_model([img_path], prompt, model, tokenizer, image_processor, context_len, conv_mode="phi", max_new_tokens=1024)

执行inference_model函数时报错--------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/qa-caif-cicd/renzhifei/xyz_pdf/mPLUG-DocOwl/TinyChart/inference.ipynb Cell 6 line 1 ----> 1 response = inference_model([img_path], prompt, model, tokenizer, image_processor, context_len, conv_mode="phi", max_new_tokens=1024)

File ~/renzhifei/xyz_pdf/mPLUG-DocOwl/TinyChart/tinychart/eval/run_tiny_chart.py:88, in inference_model(image_files, query, model, tokenizer, image_processor, context_len, conv_mode, temperature, max_new_tokens) 85 stopping_criteria = KeywordsStoppingCriteria(keywords, tokenizer, input_ids) 87 with torch.inference_mode(): ---> 88 output_ids = model.generate( 89 input_ids, 90 images=images_tensor, 91 do_sample=True if temperature > 0 else False, 92 temperature=temperature, 93 # top_p=top_p, 94 # num_beams=args.num_beams, 95 pad_token_id=tokenizer.pad_token_id, 96 max_new_tokens=max_new_tokens, 97 use_cache=True, 98 stopping_criteria=[stopping_criteria], 99 ) 101 outputs = tokenizer.batch_decode( 102 output_ids, skip_special_tokens=True 103 )[0] 104 outputs = outputs.strip() ... -> 1520 return forward_call(*args, **kwargs) 1522 try: 1523 result = None

TypeError: TinyChartPhiForCausalLM.forward() got an unexpected keyword argument 'cache_position'`

[Willie542]

I also have the same problem, have you solved it?

[chaserRen]

I also have the same problem, have you solved it?

No, I have not been able to solve this problem.

[abdellah-Taibaoui]

I also have this problem, have you solved it ?

[ivanluk914]

transformers==4.37.2 works for me https://github.com/haotian-liu/LLaVA/issues/1218

[Qigqi]

def forward( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = None, position_ids: Optional[torch.LongTensor] = None, past_key_values: Optional[List[torch.FloatTensor]] = None, inputs_embeds: Optional[torch.FloatTensor] = None, labels: Optional[torch.LongTensor] = None, use_cache: Optional[bool] = None, output_attentions: Optional[bool] = None, output_hidden_states: Optional[bool] = None, images: Optional[torch.FloatTensor] = None, return_dict: Optional[bool] = None, cache_position: Optional[bool] = None, )https://github.com/X-PLUG/mPLUG-DocOwl/blob/main/TinyChart/tinychart/model/language_model/llava_phi.py