mlc-llm icon indicating copy to clipboard operation
mlc-llm copied to clipboard

Published 20 hours ago • verified publisher icon mlc-ai

What is the best way to deploy the 'mlc_chat' service as an API that accepts HTTP requests?

Open NeoWang9999 opened this issue 2 years ago • 2 comments

I tested the 'mlc_chat_cli' command in a Linux environment, and I want to deploy it as an API that can call services. Is there a convenient way to deploy it, such as a Docker image?

NeoWang9999 avatar May 18 '23 04:05 NeoWang9999

Not currently, but we are in the process of putting up a PR soon that will expose a REST API.

sudeepag avatar May 23 '23 03:05 sudeepag

Please check out https://github.com/mlc-ai/mlc-llm/tree/main/python#rest-api for instructions on how to use the REST API.

sudeepag avatar Jun 09 '23 02:06 sudeepag

closing as rest api is now part of the main

tqchen avatar Jun 16 '23 15:06 tqchen