LLaMA-Factory icon indicating copy to clipboard operation
LLaMA-Factory copied to clipboard

Published 20 hours ago • verified publisher icon hiyouga

Is api module support function calling?

Open Stephen-SMJ opened this issue 9 months ago • 1 comments

Reminder

  • [X] I have read the README and searched the existing issues.

Reproduction

I use the script: llamafactory-cli api --model_name_or_path Qwen/Qwen1.5-7B-Chat --template qwen deploy many models like qwen1.5_7b_chat, Baichuan2-7B-Chat, Trelis/Meta-Llama-3-8B-Instruct-function-calling and so on. However I found It does not give a function calling result although I passed the functions. So I want to confirm is this API module supports function calling?

Expected behavior

Is it support function calling?

System Info

No response

Others

No response

Stephen-SMJ avatar May 06 '24 08:05 Stephen-SMJ

Maybe you can try this script https://github.com/hiyouga/LLaMA-Factory/blob/main/tests/test_toolcall.py for a reference.

codemayq avatar May 06 '24 09:05 codemayq

This api only supports the models finetuned by llama factory on the function call dataset, it may not support the external models

hiyouga avatar May 06 '24 16:05 hiyouga