inference
inference copied to clipboard
Published
4 weeks ago
•
xorbitsai
xorbitsai
QUESTION: `xinference registrations` command error: requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
the version is xinference 0.9.2 ,when inspect the registered LLM using 'xinference registrations --model-type LLM', an error occurred
File "/home/ubuntu/anaconda3/envs/xinference092/bin/xinference", line 8, in <module>
sys.exit(cli())
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 447, in list_model_registrations
client = RESTfulClient(base_url=endpoint)
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 658, in __init__
self._check_cluster_authenticated()
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 681, in _check_cluster_authenticated
f"Failed to get cluster information, detail: {response.json()['detail']}"
File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```
the version of package is
----------------------------- ------------
accelerate 0.28.0
addict 2.4.0
aiobotocore 2.7.0
aiofiles 23.2.1
aiohttp 3.9.3
aioitertools 0.11.0
aioprometheus 23.12.0
aiosignal 1.3.1
aliyun-python-sdk-core 2.15.0
aliyun-python-sdk-kms 2.16.2
altair 5.2.0
annotated-types 0.6.0
anyio 4.3.0
async-timeout 4.0.3
attrs 23.2.0
auto_gptq 0.7.1
bcrypt 4.1.2
bitsandbytes 0.43.0
botocore 1.31.64
certifi 2024.2.2
cffi 1.16.0
charset-normalizer 3.3.2
chatglm-cpp 0.3.1
click 8.1.7
cloudpickle 3.0.0
colorama 0.4.6
coloredlogs 15.0.1
contourpy 1.2.0
controlnet-aux 0.0.7
crcmod 1.7
cryptography 42.0.5
cycler 0.12.1
datasets 2.18.0
diffusers 0.26.3
dill 0.3.8
diskcache 5.6.3
distro 1.9.0
ecdsa 0.18.0
einops 0.7.0
exceptiongroup 1.2.0
fastapi 0.110.0
ffmpy 0.3.2
filelock 3.13.1
fonttools 4.49.0
frozenlist 1.4.1
fsspec 2023.10.0
gast 0.5.4
gekko 1.0.7
gradio 4.21.0
gradio_client 0.12.0
h11 0.14.0
httpcore 1.0.4
httptools 0.6.1
httpx 0.27.0
huggingface-hub 0.21.4
humanfriendly 10.0
idna 3.6
imageio 2.34.0
importlib_metadata 7.0.2
importlib_resources 6.3.0
Jinja2 3.1.3
jmespath 0.10.0
joblib 1.3.2
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
kiwisolver 1.4.5
lazy_loader 0.3
llama_cpp_python 0.2.56
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.8.3
mdurl 0.1.2
modelscope 1.13.1
mpmath 1.3.0
msgpack 1.0.8
multidict 6.0.5
multiprocess 0.70.16
networkx 3.2.1
ninja 1.11.1.1
numpy 1.26.4
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.4.99
nvidia-nvtx-cu12 12.1.105
openai 1.13.3
opencv-python 4.9.0.80
optimum 1.17.1
orjson 3.9.15
oss2 2.18.4
packaging 24.0
pandas 2.2.1
passlib 1.7.4
peft 0.9.0
pillow 10.2.0
pip 23.3.1
platformdirs 4.2.0
protobuf 4.25.3
psutil 5.9.8
pyarrow 15.0.1
pyarrow-hotfix 0.6
pyasn1 0.5.1
pycparser 2.21
pycryptodome 3.20.0
pydantic 2.6.4
pydantic_core 2.16.3
pydub 0.25.1
Pygments 2.17.2
pynvml 11.5.0
pyparsing 3.1.2
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
python-jose 3.3.0
python-multipart 0.0.9
pytz 2024.1
PyYAML 6.0.1
quantile-python 1.1
ray 2.9.3
referencing 0.33.0
regex 2023.12.25
requests 2.31.0
rich 13.7.1
rouge 1.0.1
rpds-py 0.18.0
rsa 4.9
ruff 0.3.2
s3fs 2023.10.0
safetensors 0.4.2
scikit-image 0.22.0
scikit-learn 1.4.1.post1
scipy 1.12.0
semantic-version 2.10.0
sentence-transformers 2.5.1
sentencepiece 0.2.0
setuptools 68.2.2
shellingham 1.5.4
simplejson 3.19.2
six 1.16.0
sniffio 1.3.1
sortedcontainers 2.4.0
sse-starlette 2.0.0
starlette 0.36.3
sympy 1.12
tabulate 0.9.0
tblib 3.0.0
threadpoolctl 3.3.0
tifffile 2024.2.12
tiktoken 0.6.0
timm 0.9.16
tokenizers 0.15.2
tomli 2.0.1
tomlkit 0.12.0
toolz 0.12.1
torch 2.1.2
torchvision 0.16.2
tqdm 4.66.2
transformers 4.38.2
transformers-stream-generator 0.0.5
triton 2.1.0
typer 0.9.0
typing_extensions 4.10.0
tzdata 2024.1
urllib3 2.0.7
uvicorn 0.28.0
uvloop 0.19.0
vllm 0.3.0
watchfiles 0.21.0
websockets 11.0.3
wheel 0.41.2
wrapt 1.16.0
xformers 0.0.23.post1
xinference 0.9.2
xoscar 0.2.1
xxhash 3.4.1
yapf 0.40.2
yarl 1.9.4
zipp 3.18.0```
the version is xinference 0.9.2 ,when inspect the registered LLM using 'xinference registrations --model-type LLM', an error occurred
File "/home/ubuntu/anaconda3/envs/xinference092/bin/xinference", line 8, in <module> sys.exit(cli()) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 447, in list_model_registrations client = RESTfulClient(base_url=endpoint) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 658, in __init__ self._check_cluster_authenticated() File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 681, in _check_cluster_authenticated f"Failed to get cluster information, detail: {response.json()['detail']}" File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```
Does the xinference start normally on port 9997?
the version is xinference 0.9.2 ,when inspect the registered LLM using 'xinference registrations --model-type LLM', an error occurred
File "/home/ubuntu/anaconda3/envs/xinference092/bin/xinference", line 8, in <module> sys.exit(cli()) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/deploy/cmdline.py", line 447, in list_model_registrations client = RESTfulClient(base_url=endpoint) File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 658, in __init__ self._check_cluster_authenticated() File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 681, in _check_cluster_authenticated f"Failed to get cluster information, detail: {response.json()['detail']}" File "/home/ubuntu/anaconda3/envs/xinference092/lib/python3.10/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)```Does the xinference start normally on port 9997?
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO Starting Xinference at endpoint: http://0.0.0.0:9997```
everything seems ok
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997
2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO Xinference supervisor 0.0.0.0:55416 started
2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO Starting metrics export server at 0.0.0.0:None
2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO Checking metrics export server...
2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO Metrics server is started at: http://0.0.0.0:34957
2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO Xinference worker 0.0.0.0:55416 started
2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO Purge cache directory: /home/ubuntu/.xinference/cache
2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO Starting Xinference at endpoint: http://0.0.0.0:9997
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997 2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO Xinference supervisor 0.0.0.0:55416 started 2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO Starting metrics export server at 0.0.0.0:None 2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO Checking metrics export server... 2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO Metrics server is started at: http://0.0.0.0:34957 2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO Xinference worker 0.0.0.0:55416 started 2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO Purge cache directory: /home/ubuntu/.xinference/cache 2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO Starting Xinference at endpoint: http://0.0.0.0:9997
Have you tried other commands? Do all commands run with this problem?
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997 2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO Xinference supervisor 0.0.0.0:55416 started 2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO Starting metrics export server at 0.0.0.0:None 2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO Checking metrics export server... 2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO Metrics server is started at: http://0.0.0.0:34957 2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO Xinference worker 0.0.0.0:55416 started 2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO Purge cache directory: /home/ubuntu/.xinference/cache 2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO Starting Xinference at endpoint: http://0.0.0.0:9997Have you tried other commands? Do all commands run with this pr
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
@BaiMeiyingxue Cannot reproduce this situation on my ubuntu machine. Could you please paste your command that is used for starting xinference?
(xinference) (base) ubuntu@ubuntu-EG540-G30:~$ xinference-local --host 0.0.0.0 --port 9997 2024-03-14 14:19:56,218 xinference.core.supervisor 2424703 INFO Xinference supervisor 0.0.0.0:55416 started 2024-03-14 14:19:57,414 xinference.core.worker 2424703 INFO Starting metrics export server at 0.0.0.0:None 2024-03-14 14:19:57,418 xinference.core.worker 2424703 INFO Checking metrics export server... 2024-03-14 14:19:59,950 xinference.core.worker 2424703 INFO Metrics server is started at: http://0.0.0.0:34957 2024-03-14 14:19:59,951 xinference.core.worker 2424703 INFO Xinference worker 0.0.0.0:55416 started 2024-03-14 14:19:59,952 xinference.core.worker 2424703 INFO Purge cache directory: /home/ubuntu/.xinference/cache 2024-03-14 14:20:02,775 xinference.api.restful_api 2424453 INFO Starting Xinference at endpoint: http://0.0.0.0:9997Have you tried other commands? Do all commands run with this problem?
yes, I tried to register a model with commands, and the seem error occured
If you have enabled a proxy, please try disabling it and then attempt the operation again. This maybe resolve the issue you encountered.
This issue is stale because it has been open for 7 days with no activity.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue was closed because it has been inactive for 5 days since being marked as stale.