gradio gradio server recursion error triggered by gradio client

trafficstars

Describe the bug

    encoded_value = jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 148, in jsonable_encoder
    if isinstance(obj, classes_tuple):
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/abc.py", line 119, in __instancecheck__
    return _abc_instancecheck(cls, instance)
RecursionError: maximum recursion depth exceeded in comparison

Only recursion error I've seen mentioned is: https://github.com/gradio-app/gradio/issues/1431

Is there an existing issue for this?

[X] I have searched the existing issues

Reproduction

Just using old gradio client 0.0.8 works fine for exactly same server and client code.

The code is not yet open-sourced, but I'll update issue if I can create a MRE.

client


from gradio_client import Client
client = Client(host)

def app_client_basic():
    instruction = "Who are you?"
    args = [instruction]

    api_name = '/submit'
    res = client.predict(
        *tuple(args),
        api_name=api_name,
    )
    print(md_to_text(res))

app_client_basic()

Screenshot

No response

Logs

This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/applications.py", line 276, in __call__
    await super().__call__(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/routing.py", line 255, in app
    content = await serialize_response(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/routing.py", line 152, in serialize_response
    return jsonable_encoder(response_content)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 131, in jsonable_encoder
    jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 161, in jsonable_encoder
    return jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 161, in jsonable_encoder
    return jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 131, in jsonable_encoder

...

  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 131, in jsonable_encoder
    jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 161, in jsonable_encoder
    return jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 148, in jsonable_encoder
    if isinstance(obj, classes_tuple):
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/abc.py", line 119, in __instancecheck__
    return _abc_instancecheck(cls, instance)
RecursionError: maximum recursion depth exceeded in comparison
Keyboard interruption in main thread... closing server.
Killing tunnel 0.0.0.0:7860 <> https://cb3c9872ed571ab557.gradio.live



### System Info

```shell
gradio==3.27.0
gradio_client==0.1.3
chrome (but not relevant, just client call)

Severity

blocking upgrade to latest gradio version

Apr 18 '23 22:04 pseudotensor

Maybe I had old server and new client.

Apr 18 '23 22:04 pseudotensor

Sorry about that @pseudotensor ! But glad you figured it out.

Apr 24 '23 14:04 freddyaboulton

I actually still see it sometimes, both so far not fatal.

Apr 24 '23 15:04 pseudotensor

Let me re-open since still occurs. We have life HF spaces at: https://huggingface.co/spaces/h2oai/h2ogpt-chatbot2

And in logs you will see:

  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 161, in jsonable_encoder
    return jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 131, in jsonable_encoder
    jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 161, in jsonable_encoder
    return jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 161, in jsonable_encoder
    return jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 131, in jsonable_encoder
    jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 161, in jsonable_encoder
    return jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/site-packages/fastapi/encoders.py", line 148, in jsonable_encoder
    if isinstance(obj, classes_tuple):
  File "/home/user/.pyenv/versions/3.8.9/lib/python3.8/abc.py", line 98, in __instancecheck__
    return _abc_instancecheck(cls, instance)
RecursionError: maximum recursion depth exceeded in comparison

The space is public, so full source code is public. The way to trigger that issue is also public: https://github.com/h2oai/h2ogpt/blob/fe1b1ae63d267c4a6021505b2f82c94247fe6c58/client_test.py#L20-L26

i.e. just hit that space with:

from gradio_client import Client
client = Client("https://h2oai-h2ogpt-chatbot2.hf.space")
args = ['', '', '', False, 'human_bot', 0.1, 0.75, 40, 1, 50, 0, False, 20, 1.0, 1, True, False, 'Who are you?', '']
res = client.predict(*tuple(args), api_name='/submit_nochat')
print(res)

then despite giving back a response fine, the logs are spammed with a massive recursion message.

Apr 24 '23 22:04 pseudotensor

@freddyaboulton Thanks for checking on this! While non-fatal, it seems like something is seriously wrong.

Apr 24 '23 22:04 pseudotensor

Ok - will take a look!

Apr 24 '23 22:04 freddyaboulton

Despite the space being public, I won’t be able to see the logs unless I’m in the same org. Can you provide the full log snippet or the part that shows which gradio line is causing the stack trace? Thanks!

Apr 24 '23 22:04 freddyaboulton

Hi @freddyaboulton , I shared above partial logs.

I don't see option to download logs from spaces. But I can copy paste for now.

spaces2.log

However, it's still truncated because seems only last N lines is every visible. I don't see how to get full logs.

My log fragment is best, since I show top and bottom: https://github.com/gradio-app/gradio/issues/3906#issue-1673890858

i.e. it starts with the below and then goes into recursion:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/applications.py", line 276, in __call__
    await super().__call__(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/routing.py", line 255, in app
    content = await serialize_response(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/routing.py", line 152, in serialize_response
    return jsonable_encoder(response_content)
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 117, in jsonable_encoder
    encoded_value = jsonable_encoder(
  File "/home/jon/miniconda3/envs/alpaca/lib/python3.10/site-packages/fastapi/encoders.py", line 131, in jsonable_encoder
    jsonable_encoder(

Apr 24 '23 23:04 pseudotensor

I was debugging an issue with the same symptoms today. As it turns out, the problem was that I was returning a Gradio component itself from an event listener instead of the value I was trying to assign.

That is, something like this:

with gr.Blocks() as demo:
    status_box = gr.Textbox()
    def test(args):
        return {status_box: status_box} # oops
        # return {status_box: args[status_box]}
    gr.Button("TEST").click(
        fn=test, 
        inputs={status_box}
        outputs={status_box}
    )

I'm not sure why exactly this leads to infinite recursion.

I'm surprised that jsonable_encoder doesn't detect recursive values, however. Even the builtin print does.

Apr 25 '23 06:04 nonnull-ca

I get my problem even when I hit the server with a random api_name. So it's not related specifically to a specific click task for me. i.e. this causes same problem:

from gradio_client import Client
client = Client("https://h2oai-h2ogpt-chatbot2.hf.space")
args = []
res = client.predict(*tuple(args), api_name='/foobar')
print(res)

Apr 25 '23 08:04 pseudotensor

I'm not sure what's happening - I do see the infinite recursion error but from what I can tell that only happened once and I'm not sure if it's triggered by the client.

After hitting the /submit_nochat endpoint, I just see Using pad_token, but it is not set yet. being printed.

h2ogpt

Apr 25 '23 16:04 freddyaboulton

Yes, you can ignore Using pad_token, but it is not set yet. -- that is just torch issue with our code.

We see it happen every time there is API call, I think, even "foobar" name, doesn't have to be real api name. But if you see it only once, I hope that's good enough.

Apr 26 '23 21:04 pseudotensor

@freddyaboulton any word?

May 27 '23 08:05 pseudotensor

Hi @pseudotensor !

I cloned your space and submitted three requests via the client. I see the "infinite recursion" but only when I created the client. Sending requests does not trigger that error. In this screenshot you can see that infinite recursion error but only once and you can see the "Using pad token" three times, which makes me think it's not triggered per request.

Will look at what could be causing this when the client connects (maybe something related to the view_api info) but seems low priority given that the app and client predictions are working well

Jun 05 '23 05:06 freddyaboulton

gradio gradio copied to clipboard

gradio server recursion error triggered by gradio client

Describe the bug

Is there an existing issue for this?

Reproduction

Screenshot

Logs

Severity

gradio
gradio copied to clipboard