Robin Zheng

2024-02-15 12:30:55,450 xinference.api.restful_api 772 ERROR [address=127.0.0.1:46891, pid=784] Model not found in the model list, uid: 3dc22fb0-740c-11ee-be13-452b73d8be98 Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/xinference/api/restful_api.py", line 542, in describe_model data = await...

If I launch model with xinference-local, why I get those kind of distributed deployment stuff. @log_async(logger=logger) async def describe_model(self, model_uid: str) -> Dict[str, Any]: replica_info = self._model_uid_to_replica_info.get(model_uid, None) if replica_info...

from xinference.client import RESTfulClient client = RESTfulClient("http://127.0.0.1:9997") print(client.list_models()) error out info: {'zephyr-7b-beta': {'model_type': 'LLM', 'address': '127.0.0.1:39873', 'accelerators': ['0', '1'], 'model_name': 'zephyr-7b-beta', 'model_lang': ['en'], 'model_ability': ['chat'], 'model_description': 'Zephyr-7B-β is the second...