Aaron Pham

This seems like a users issue, mistral is already supported by transformers, so no need to use `TRUST_REMOTE_CODE=True`.

llama 3 is already supported.

Hi there, sorry for the late response. with the 0.5 release, we are deprecating the Python API and will treat all Python SDK going forward as internal. WIth that being...

Hi there, model_id is not supposed to be used like this. The file that you pass in here is a shard of the model, meaning we won't know how to...

should be supported on main now. Will release a new version soon.

just a headsup for 0.5 I'm removing pytorch backend, will work on CPU specific branch for better optimization.

Will work on it this week.

This is the PR https://github.com/bentoml/OpenLLM/pull/821

In 0.5, you can save it to bento model store first with ```python with bentoml.models.create("your-model") as model: ... ``` then you can start with `openllm start your-model`.

@jianshen92 can you help take a look with this?