FastChat
FastChat copied to clipboard
Published
20 hours ago •
lm-sys
lm-sys
Gradio UI 0.2.36 not launching
I am running the gradio web UI 0.2.36 in a air-gapped Kubernetes cluster. Gradio app cannot start.
Version 0.2.34 used to work without issues.
I tried the following:
- setting no_proxy and NO_PROXY environment.
- trying to curl and test from within the container: no connectivity issues, as you can see the model list can be fetched from the controller service.
Here is the error log;
2024-04-08 13:53:53 | INFO | gradio_web_server | args: Namespace(host='0.0.0.0', port=8054, share=False, controller_url='http://fastchat-controller-service.fastchat.svc.cluster.local:8000', concurrency_count=10, model_list_mode='once', moderate=False, show_terms_of_use=False, register_api_endpoint_file=None, gradio_auth_path=None, gradio_root_path='/')
2024-04-08 13:53:53 | INFO | gradio_web_server | All models: ['gemma-7b-it-vllm', 'mistral-7b-16k-rtx4090', 'mistral-7b-instruct-v0.2-vllm-prod', 'mistral-7b-sq-8k-a40']
2024-04-08T13:53:53.283826411Z 2024-04-08 13:53:53 | INFO | gradio_web_server | Visible models: ['gemma-7b-it-vllm', 'mistral-7b-16k-rtx4090', 'mistral-7b-instruct-v0.2-vllm-prod', 'mistral-7b-sq-8k-a40']
2024-04-08 13:53:56 | INFO | stdout | Running on local URL: http://0.0.0.0:8054
2024-04-08 13:53:59 | ERROR | stderr | Traceback (most recent call last):
2024-04-08T13:53:59.455180148Z 2024-04-08 13:53:59 | ERROR | stderr | File "<frozen runpy>", line 198, in _run_module_as_main
2024-04-08T13:53:59.455225194Z 2024-04-08 13:53:59 | ERROR | stderr | File "<frozen runpy>", line 88, in _run_code
2024-04-08T13:53:59.455285491Z 2024-04-08 13:53:59 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/fastchat/serve/gradio_web_server.py", line 880, in <module>
2024-04-08 13:53:59 | ERROR | stderr | ).launch(
2024-04-08T13:53:59.455700341Z 2024-04-08 13:53:59 | ERROR | stderr | ^^^^^^^
2024-04-08T13:53:59.455767730Z 2024-04-08 13:53:59 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/gradio/blocks.py", line 2210, in launch
2024-04-08 13:53:59 | ERROR | stderr | raise ValueError(
2024-04-08T13:53:59.456269490Z 2024-04-08 13:53:59 | ERROR | stderr | ValueError: When localhost is not accessible, a shareable link must be created. Please set share=True or check your proxy settings to allow access to localhost.
2024-04-08 13:54:13 | ERROR | stderr | Exception in thread Thread-3 (_do_normal_analytics_request):
2024-04-08T13:54:13.544610255Z 2024-04-08 13:54:13 | ERROR | stderr | Traceback (most recent call last):
2024-04-08T13:54:13.544615226Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
2024-04-08 13:54:13 | ERROR | stderr | yield
2024-04-08T13:54:13.544848850Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 233, in handle_request
2024-04-08T13:54:13.544975567Z 2024-04-08 13:54:13 | ERROR | stderr | resp = self._pool.handle_request(req)
2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-08T13:54:13.549636099Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
2024-04-08T13:54:13.549638666Z 2024-04-08 13:54:13 | ERROR | stderr | raise exc from None
2024-04-08T13:54:13.549640682Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
2024-04-08T13:54:13.549642685Z 2024-04-08 13:54:13 | ERROR | stderr | response = connection.handle_request(
2024-04-08T13:54:13.549644505Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 99, in handle_request
2024-04-08T13:54:13.549648193Z 2024-04-08 13:54:13 | ERROR | stderr | raise exc
2024-04-08T13:54:13.549655590Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 76, in handle_request
2024-04-08 13:54:13 | ERROR | stderr | stream = self._connect(request)
2024-04-08T13:54:13.549659265Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^
2024-04-08T13:54:13.549661013Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 122, in _connect
2024-04-08T13:54:13.549662962Z 2024-04-08 13:54:13 | ERROR | stderr | stream = self._network_backend.connect_tcp(**kwargs)
2024-04-08T13:54:13.549664798Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpcore/_backends/sync.py", line 205, in connect_tcp
2024-04-08T13:54:13.549992493Z 2024-04-08 13:54:13 | ERROR | stderr | with map_exceptions(exc_map):
2024-04-08T13:54:13.549995046Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
2024-04-08T13:54:13.549996845Z 2024-04-08 13:54:13 | ERROR | stderr | self.gen.throw(typ, value, traceback)
2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
2024-04-08T13:54:13.550000771Z 2024-04-08 13:54:13 | ERROR | stderr | raise to_exc(exc) from exc
2024-04-08T13:54:13.550002574Z 2024-04-08 13:54:13 | ERROR | stderr | httpcore.ConnectTimeout: timed out
2024-04-08T13:54:13.550004400Z 2024-04-08 13:54:13 | ERROR | stderr |
2024-04-08 13:54:13 | ERROR | stderr | The above exception was the direct cause of the following exception:
2024-04-08T13:54:13.550007959Z 2024-04-08 13:54:13 | ERROR | stderr |
2024-04-08T13:54:13.550009676Z 2024-04-08 13:54:13 | ERROR | stderr | Traceback (most recent call last):
2024-04-08T13:54:13.550011625Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
2024-04-08 13:54:13 | ERROR | stderr | self.run()
2024-04-08T13:54:13.550015174Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/threading.py", line 982, in run
2024-04-08T13:54:13.550016913Z 2024-04-08 13:54:13 | ERROR | stderr | self._target(*self._args, **self._kwargs)
2024-04-08T13:54:13.550018698Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/gradio/analytics.py", line 61, in _do_normal_analytics_request
2024-04-08T13:54:13.550020904Z 2024-04-08 13:54:13 | ERROR | stderr | data["ip_address"] = get_local_ip_address()
2024-04-08T13:54:13.550022609Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^
2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/gradio/analytics.py", line 117, in get_local_ip_address
2024-04-08 13:54:13 | ERROR | stderr | ip_address = httpx.get(
2024-04-08T13:54:13.550027685Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^
2024-04-08T13:54:13.550029409Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_api.py", line 198, in get
2024-04-08T13:54:13.550031144Z 2024-04-08 13:54:13 | ERROR | stderr | return request(
2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^
2024-04-08T13:54:13.550034609Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_api.py", line 106, in request
2024-04-08T13:54:13.550064542Z 2024-04-08 13:54:13 | ERROR | stderr | return client.request(
2024-04-08T13:54:13.550066532Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^
2024-04-08T13:54:13.550068277Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 827, in request
2024-04-08T13:54:13.550070033Z 2024-04-08 13:54:13 | ERROR | stderr | return self.send(request, auth=auth, follow_redirects=follow_redirects)
2024-04-08T13:54:13.550077757Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 914, in send
2024-04-08T13:54:13.550081399Z 2024-04-08 13:54:13 | ERROR | stderr | response = self._send_handling_auth(
2024-04-08T13:54:13.550083109Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-08T13:54:13.550084873Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 942, in _send_handling_auth
2024-04-08T13:54:13.550086579Z 2024-04-08 13:54:13 | ERROR | stderr | response = self._send_handling_redirects(
2024-04-08T13:54:13.550088270Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-08T13:54:13.550090121Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
2024-04-08T13:54:13.550091822Z 2024-04-08 13:54:13 | ERROR | stderr | response = self._send_single_request(request)
2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-08T13:54:13.550096950Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1015, in _send_single_request
2024-04-08T13:54:13.550098853Z 2024-04-08 13:54:13 | ERROR | stderr | response = transport.handle_request(request)
2024-04-08T13:54:13.550100619Z 2024-04-08 13:54:13 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-08T13:54:13.550102442Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 232, in handle_request
2024-04-08T13:54:13.550104155Z 2024-04-08 13:54:13 | ERROR | stderr | with map_httpcore_exceptions():
2024-04-08T13:54:13.550105952Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
2024-04-08T13:54:13.550107787Z 2024-04-08 13:54:13 | ERROR | stderr | self.gen.throw(typ, value, traceback)
2024-04-08T13:54:13.550109610Z 2024-04-08 13:54:13 | ERROR | stderr | File "/usr/local/lib/python3.11/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions
2024-04-08T13:54:13.550111320Z 2024-04-08 13:54:13 | ERROR | stderr | raise mapped_exc(message) from exc
2024-04-08T13:54:13.550113006Z 2024-04-08 13:54:13 | ERROR | stderr | httpx.ConnectTimeout: timed out
pip list
Package Version
------------------------- -----------
accelerate 0.29.1
aiofiles 23.2.1
aiohttp 3.9.3
aiosignal 1.3.1
altair 5.3.0
annotated-types 0.6.0
anyio 4.3.0
attrs 23.2.0
certifi 2024.2.2
charset-normalizer 3.3.2
click 8.1.7
contourpy 1.2.1
cycler 0.12.1
fastapi 0.110.1
ffmpy 0.3.2
filelock 3.13.3
fonttools 4.51.0
frozenlist 1.4.1
fschat 0.2.36
fsspec 2024.3.1
gradio 4.25.0
gradio_client 0.15.0
h11 0.14.0
httpcore 1.0.5
httpx 0.27.0
huggingface-hub 0.22.2
idna 3.6
importlib_resources 6.4.0
Jinja2 3.1.3
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
kiwisolver 1.4.5
markdown-it-py 3.0.0
markdown2 2.4.13
MarkupSafe 2.1.5
matplotlib 3.8.4
mdurl 0.1.2
mpmath 1.3.0
multidict 6.0.5
networkx 3.3
nh3 0.2.17
numpy 1.26.4
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.19.3
nvidia-nvjitlink-cu12 12.4.127
nvidia-nvtx-cu12 12.1.105
orjson 3.10.0
packaging 24.0
pandas 2.2.1
peft 0.10.0
pillow 10.3.0
pip 24.0
prompt-toolkit 3.0.43
protobuf 5.26.1
psutil 5.9.8
pydantic 2.6.4
pydantic_core 2.16.3
pydub 0.25.1
Pygments 2.17.2
pyparsing 3.1.2
python-dateutil 2.9.0.post0
python-multipart 0.0.9
pytz 2024.1
PyYAML 6.0.1
referencing 0.34.0
regex 2023.12.25
requests 2.31.0
rich 13.7.1
rpds-py 0.18.0
ruff 0.3.5
safetensors 0.4.2
semantic-version 2.10.0
sentencepiece 0.2.0
setuptools 65.5.1
shellingham 1.5.4
shortuuid 1.0.13
six 1.16.0
sniffio 1.3.1
starlette 0.37.2
svgwrite 1.4.3
sympy 1.12
tiktoken 0.6.0
tokenizers 0.15.2
tomlkit 0.12.0
toolz 0.12.1
torch 2.2.2
tqdm 4.66.2
transformers 4.39.3
triton 2.2.0
typer 0.12.1
typing_extensions 4.11.0
tzdata 2024.1
urllib3 2.2.1
uvicorn 0.29.0
wavedrom 2.0.3.post3
wcwidth 0.2.13
websockets 11.0.3
wheel 0.43.0
yarl 1.9.4
root@fastchat-gradio-web-deployment-54f9b
If i remove the argument
--gradio-root-path /
it starts working. So this is probably a bug in gradio.