gradio
gradio copied to clipboard
Gradio client: Calling sync() from within a running loop
Describe the bug
I currently have a Flask app that uses Gradio API for inferencing. If I run it with normal Flask built-in WSGI server, it'll work. However, I want to run it with gunicorn and gevent in my server.
Have you searched existing issues? 🔎
- [x] I have searched and found no existing issues
Reproduction
from gevent import monkey
monkey.patch_all()
from gradio_client import Client
from flask import Flask, send_file
import time
app = Flask(__name__)
imageclient = Client("https://diffusers-unofficial-sdxl-turbo-i2i-t2i.hf.space/")
@app.route("/gen")
def gen():
result = imageclient.predict(
None,
"A sunny view of the park",
0.9,
2,
round(time.time()),
api_name="/predict"
)
return send_file(result)
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000)
Then run it with gunicorn -k gevent -w 1 <app_name>:app --bind 0.0.0.0:5000
Access 127.0.0.1:5000/gen
Screenshot
No response
Logs
Traceback (most recent call last):
File "/home/Joystick/.local/lib/python3.10/site-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
File "/home/Joystick/.local/lib/python3.10/site-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/Joystick/.local/lib/python3.10/site-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
File "/home/Joystick/.local/lib/python3.10/site-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
File "/home/Joystick/ConversAItionalasFlask/app.py", line 414, in create_image
result = imageclient.predict(
File "/home/Joystick/.local/lib/python3.10/site-packages/gradio_client/client.py", line 389, in predict
return self.submit(*args, api_name=api_name, fn_index=fn_index).result()
File "/home/Joystick/.local/lib/python3.10/site-packages/gradio_client/client.py", line 1590, in result
return super().result(timeout=timeout)
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/Joystick/.local/lib/python3.10/site-packages/gradio_client/client.py", line 973, in _inner
predictions = _predict(*data)
File "/home/Joystick/.local/lib/python3.10/site-packages/gradio_client/client.py", line 999, in _predict
result = utils.synchronize_async(
File "/home/Joystick/.local/lib/python3.10/site-packages/gradio_client/utils.py", line 870, in synchronize_async
return fsspec.asyn.sync(fsspec.asyn.get_loop(), func, *args, **kwargs) # type: ignore
File "/home/Joystick/.local/lib/python3.10/site-packages/fsspec/asyn.py", line 80, in sync
raise NotImplementedError("Calling sync() from within a running loop")
NotImplementedError: Calling sync() from within a running loop
System Info
Joystick@Wah:~/ConversAItionalasFlask$ pip install gradio_client
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: gradio_client in /home/Joystick/.local/lib/python3.10/site-packages (0.9.0)
Requirement already satisfied: packaging in /home/Joystick/.local/lib/python3.10/site-packages (from gradio_client) (23.2)
Requirement already satisfied: httpx in /home/Joystick/.local/lib/python3.10/site-packages (from gradio_client) (0.26.0)
Requirement already satisfied: websockets<12.0,>=10.0 in /home/Joystick/.local/lib/python3.10/site-packages (from gradio_client) (11.0.3)
Requirement already satisfied: fsspec in /home/Joystick/.local/lib/python3.10/site-packages (from gradio_client) (2024.2.0)
Requirement already satisfied: huggingface-hub>=0.19.3 in /home/Joystick/.local/lib/python3.10/site-packages (from gradio_client) (0.20.3)
Requirement already satisfied: typing-extensions~=4.0 in /home/Joystick/.local/lib/python3.10/site-packages (from gradio_client) (4.9.0)
Requirement already satisfied: pyyaml>=5.1 in /usr/lib/python3/dist-packages (from huggingface-hub>=0.19.3->gradio_client) (5.4.1)
Requirement already satisfied: filelock in /home/Joystick/.local/lib/python3.10/site-packages (from huggingface-hub>=0.19.3->gradio_client) (3.13.1)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from huggingface-hub>=0.19.3->gradio_client) (2.25.1)
Requirement already satisfied: tqdm>=4.42.1 in /home/Joystick/.local/lib/python3.10/site-packages (from huggingface-hub>=0.19.3->gradio_client) (4.66.1)
Requirement already satisfied: anyio in /home/Joystick/.local/lib/python3.10/site-packages (from httpx->gradio_client) (4.2.0)
Requirement already satisfied: httpcore==1.* in /home/Joystick/.local/lib/python3.10/site-packages (from httpx->gradio_client) (1.0.2)
Requirement already satisfied: idna in /usr/lib/python3/dist-packages (from httpx->gradio_client) (3.3)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from httpx->gradio_client) (2020.6.20)
Requirement already satisfied: sniffio in /home/Joystick/.local/lib/python3.10/site-packages (from httpx->gradio_client) (1.3.0)
Requirement already satisfied: h11<0.15,>=0.13 in /home/Joystick/.local/lib/python3.10/site-packages (from httpcore==1.*->httpx->gradio_client) (0.14.0)
Requirement already satisfied: exceptiongroup>=1.0.2 in /home/Joystick/.local/lib/python3.10/site-packages (from anyio->httpx->gradio_client) (1.2.0)
Severity
Blocking usage of gradio
Has there been any updates to this? Thanks
Sorry not yet @Joystickplays
When using gradio_client in flask api and setting gunicorn parameter worker_class='eventlet', an error will occur.
Will require some refactoring of the client that we are planning as part of the 1.0 release