[Question]: tiny-llama可以转成pdparams格式，可以加载，但无法推理出有效结果

[1826133674]

请提出你的问题

完成tiny-llama的convert后，我修改了tokenizer.model的名字后，可以成功加载但是，推理不出正确的结果。 我的环境： paddle-bfloat 0.1.7 paddle2onnx 1.1.0 paddlefsl 1.1.0 paddlenlp 2.7.0.post0 paddlepaddle-gpu 2.6.0.post112 转换代码 from paddlenlp.transformers import AutoModelForCausalLM

AutoModelForCausalLM.from_pretrained("./TinyLlama-1.1B-Chat-v1.0", convert_from_torch=True, dtype="float16")

推理代码 from paddlenlp.transformers import AutoModelForCausalLM, AutoTokenizer paddle.set_device("gpu:3") print("paddle.device",paddle.get_device()) model = AutoModelForCausalLM.from_pretrained("./TinyLlama-1.1B-Chat-v1.0") tokenizer = AutoTokenizer.from_pretrained("./TinyLlama-1.1B-Chat-v1.0") input_features = tokenizer("introduce your self!", return_tensors="pd") outputs = model.generate(**input_features, max_new_tokens=128) print(outputs) ans = tokenizer.batch_decode(outputs[0]) print("ans",ans)

推理结果

W0130 10:45:50.950965 55792 dygraph_functions.cc:52641] got different data type, run type protmotion automatically, this may cause data type been changed. (Tensor(shape=[1, 1], dtype=int64, place=Place(gpu:3), stop_gradient=True, [[2]]), Tensor(shape=[1, 1], dtype=float32, place=Place(gpu:3), stop_gradient=True, [[12.16406250]])) ans ['']

[wawltor]

使用Pytorch使用同样的query可以正常预测吗？

[github-actions[bot]]

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动，被标记为stale。