"Event not found in queue" when deploying Gradio behind a proxy

[shaojun]

Describe the bug

followed exactly the official tutorial, the web page can show correctly, but if I entered sth in textbox, and click submit, the exception show:

(base) shao@shaothinkbook:~/ctrl-robotic-via-gpt$  cd /home/shao/ctrl-robotic-via-gpt ; /usr/bin/env /home/shao/miniconda3/envs/ctrl-robotic/bin/python /home/shao/.vscode-server/extensions/ms-python.debugpy-2024.4.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher 45695 -- /home/shao/ctrl-robotic-via-gpt/pc_host_main.py 
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/gradio/queueing.py", line 223, in push
    event_queue = self.event_queue_per_concurrency_id[event.concurrency_id]
KeyError: '139787653332224'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/gradio/route_utils.py", line 695, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/gradio/route_utils.py", line 711, in simple_response
    await self.app(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/gradio/routes.py", line 747, in queue_join
    return await queue_join_helper(body, request, username)
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/gradio/routes.py", line 765, in queue_join_helper
    success, event_id = await blocks._queue.push(
  File "/home/shao/miniconda3/envs/ctrl-robotic/lib/python3.10/site-packages/gradio/queueing.py", line 225, in push
    raise KeyError(
KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'

can get to work by downgrade to 4.8.0:

pip install gradio==4.8.0

Have you searched existing issues? 🔎

• [X] I have searched and found no existing issues

Reproduction

import gradio as gr


def greet(name,):
    return "Hello, " + name + "!"


demo = gr.Interface(
    fn=greet,
    inputs=[gr.Textbox(label="command to robot")],
    outputs=["text"],
)

demo.launch()

Screenshot

No response

Logs

No response

System Info

> Windows 11 wls2, ubuntu22.
> (ctrl-robotic) shao@shaothinkbook:~/ctrl-robotic-via-gpt$ pip freeze | grep gradio
> gradio==4.27.0
> gradio_client==0.15.1
> Python 3.10

Severity

Blocking usage of gradio

[abidlabs]

Hi @shaojun are you installing Gradio on multiple replicas? Or can you tell us more about how this Gradio app is deployed?

[shaojun]

Hi @shaojun are you installing Gradio on multiple replicas? Or can you tell us more about how this Gradio app is deployed?

I just run with vscode debug as started with single python file, in my laptop. I don't know what is multiple replicas.

[abidlabs]

This is super strange, I would not have expected this error. Can you try a fresh installation of gradio using pip instead of conda

cc @aliabid94 if you have any ideas

[shaojun]

I do install the gradio by pip though i am using miniconda. My ubuntu has a http and https proxy set, not sure it's related or not.

[abidlabs]

Ah yeah is it possible for you test without the proxy set?

[shaojun]

I have unset https_proxy and unset http_proxy, and then re-run and refresh the app web page, it works. so its confirmed caused by http proxy? but it does work in old version of gradio.

[Luvata]

you can try export no_proxy="localhost,127.0.0.1" before running the gradio app

[jsjolund]

I am also encountering this issue, running Gradio on Kubernetes with Ray Serve and a single replica:

(ProxyActor pid=7460)     await app(scope, receive, sender)
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
(ProxyActor pid=7460)     response = await func(request)
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
(ProxyActor pid=7460)     raw_response = await run_endpoint_function(
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
(ProxyActor pid=7460)     return await dependant.call(**values)
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/gradio/routes.py", line 747, in queue_join
(ProxyActor pid=7460)     return await queue_join_helper(body, request, username)
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/gradio/routes.py", line 765, in queue_join_helper
(ProxyActor pid=7460)     success, event_id = await blocks._queue.push(
(ProxyActor pid=7460)   File "/opt/venv/lib/python3.10/site-packages/gradio/queueing.py", line 225, in push
(ProxyActor pid=7460)     raise KeyError(
(ProxyActor pid=7460) KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'

Here is the serve config.yaml:

proxy_location: EveryNode

http_options:
  host: 0.0.0.0
  port: 8000

applications:

- name: ChatBotGroup
  route_prefix: /generate
  import_path: demo:bot
  runtime_env: {
    "working_dir": "file:///root/app/app.zip",
    "env_vars": {
      "HF_HUB_ENABLE_HF_TRANSFER": "1",
    }
  }
  deployments:
  - name: ChatBot
    user_config:
      repo_id: "TheBloke/dolphin-2.5-mixtral-8x7b-GGUF"
      model_file: "dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf"
      tensor_split: [0.6, 1, 1, 1]
    ray_actor_options:
      num_gpus: 4.0
    num_replicas: 2

- name: ChatClient
  route_prefix: /
  import_path: demo:app
  runtime_env: {
    "working_dir": "file:///root/app/app.zip",
    "env_vars": {
      "CONCURRENCY_LIMIT": "2",

      "TITLE": "Chat - Dolphin 2.5 Mixtral 8x7b",
      "CHAT_LABEL": "dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf",
    }
  }
  deployments:
  - name: ChatIngress
    num_replicas: 1

Downgrading with pip install gradio==4.24.0 fixed the issue.

[zhangyilun]

Downgrading with pip install gradio==4.24.0 fixed the issue.

This didn't work for me, but

you can try export no_proxy="localhost,127.0.0.1" before running the gradio app

This worked.

[ibrahim737701]

None of the above suggestion are working for me.

[Archer6621]

I have the same error, deploying gradio 4.27.0 on docker compose under FastAPI 0.110.3. I can access the Gradio chat example fine under localhost:8000/gradio on the host, but once I try to send a message, the same error occurs: KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'

I run FastAPI using uvicorn: uvicorn gradio_app:app --host 0.0.0.0

In gradio_app.py:

from fastapi import FastAPI, Request
import gradio as gr
import requests
import datetime
CUSTOM_PATH = "/gradio"

app = FastAPI()

def greet(name):
    return "Hello " + name + "!"

io = gr.Interface(fn=greet, inputs="textbox", outputs="textbox")
gradio_app = gr.routes.App.create_app(io)
app.mount(CUSTOM_PATH, gradio_app)

I'm setting the following environment variables in the dockerfile (from python:3-slim):

ENV GRADIO_SERVER_NAME="0.0.0.0"
ENV COMMANDLINE_ARGS="--no-gradio-queue"

I can provide a full minimal example if needed.

[iwannabewater]

you can try export no_proxy="localhost,127.0.0.1" before running the gradio app

Thanks, this works for me!

[abidlabs]

@shaojun (or anybody else who faced this issue), if you can provide more details about the proxy you're using and how it's configured, we might be able to provide a solution for this.

[iwannabewater]

@shaojun (or anybody else who faced this issue), if you can provide more details about the proxy you're using and how it's configured, we might be able to provide a solution for this. （或遇到此问题的任何其他人），如果您可以提供有关您正在使用的代理及其配置方式的更多详细信息，我们也许能够为此提供解决方案。

Environment

• Operating System: Ubuntu 22.04
• Python Version: 3.10.14
• Gradio Version: 4.29.0

Issue Description

I encountered a KeyError: 'Event not found in queue' when running a Python program that uses Gradio on an Ubuntu 22.04 server, connected via SSH from a Windows machine(Same LAN). This setup involves a reverse proxy using the following environment variables:

export http_proxy=http://<windows-ip>:<port>
export https_proxy=http://<windows-ip>:<port>

where and are the IP and port of the proxy software on Windows.

Steps to Reproduce Set up a reverse proxy using the above environment variables. Run a Python program with Gradio on the server.

Error Message

KeyError: 'Event not found in queue. If you are deploying this Gradio app with multiple replicas, please enable stickiness to ensure that all requests from the same user are routed to the same instance.'

Temporary Solution Adding export no_proxy="localhost,127.0.0.1" to the environment variables before running the Gradio app resolves the issue.

[abidlabs]

Which reverse proxy library /software are you using?

[iwannabewater]

I'm using clash for windows(CFW), can this help?

[abidlabs]

Thanks @iwannabewater I think that should be enough for us to repro this. I'll take a look!

[shaojun]

I have unset https_proxy and unset http_proxy, and then re-run and refresh the app web page, it works. so its confirmed caused by http proxy? but it does work in old version of gradio.

I have updated to gradio 4.31.1 and still see this error with http_proxy and https_proxy set. And by unset http_proxy and unset https_proxy can fix this issue.