mlc-llm icon indicating copy to clipboard operation
mlc-llm copied to clipboard

Published 20 hours ago • verified publisher icon mlc-ai

Question : Prompt configuration

Open x4080 opened this issue 2 years ago • 2 comments

Hi, I have M2 pro 16gb and it run really fast using vicuna. I wondered how to customize prompt, temperature, context etc ?

Thanks

x4080 avatar May 26 '23 03:05 x4080

To customize temperature you can modify the mlc-chat-config.json file in your model directory. Currently, you need to modify the MLC-LLM source code to customize the prompt, we will make it easier after PR #251 gets merged. Please stay tuned!

yzh119 avatar May 27 '23 22:05 yzh119

@yzh119 Thanks, I'll wait for it

x4080 avatar May 28 '23 00:05 x4080

Please refer to this page for detailed documentation: https://mlc.ai/mlc-llm/docs/tutorials/runtime/mlc_chat_config.html

junrushao avatar Jun 14 '23 05:06 junrushao

@junrushao Thanks for the documentation

x4080 avatar Jun 14 '23 21:06 x4080