port being used when restarting web ui

[catalpaaa]

Describe the bug

when restart the web ui, the port is always shown to be used. my temp solusion is to sleep 120s before recreating interface.

ie add time.sleep(120) after the very last shared.gradio['interface'].close() in server.py

it might not be a thing for newer gradio cus A1111's webui dose not have this issue. or maybe he did some blackmagic

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

start normal, change mode or do whatever that restart the webui boom

Screenshot

No response

Logs

Running on local URL:  http://0.0.0.0:9441

To create a public link, set `share=True` in `launch()`.
Task exception was never retrieved
future: <Task finished name='02liijn6knwb_18' coro=<Queue.process_events() done, defined at /home/catalpa/anaconda3/envs/textgen/lib/python3.10/site-packages/gradio/queueing.py:329> exception=AssertionError('No event data')>
Traceback (most recent call last):
  File "/home/catalpa/anaconda3/envs/textgen/lib/python3.10/site-packages/gradio/queueing.py", line 343, in process_events
    response = await self.call_prediction(awake_events, batch)
  File "/home/catalpa/anaconda3/envs/textgen/lib/python3.10/site-packages/gradio/queueing.py", line 303, in call_prediction
    assert data is not None, "No event data"
AssertionError: No event data
Closing server running on port: 9441
/home/catalpa/anaconda3/envs/textgen/lib/python3.10/site-packages/gradio/deprecation.py:40: UserWarning: The 'type' parameter has been deprecated. Use the Number component instead.
  warnings.warn(value)
Traceback (most recent call last):
  File "/home/catalpa/anaconda3/envs/textgen/lib/python3.10/site-packages/gradio/networking.py", line 119, in start_server
    s.bind((LOCALHOST_NAME, server_port))
OSError: [Errno 98] Address already in use

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/catalpa/nas/program/text-generation-webui/server.py", line 468, in <module>
    create_interface()
  File "/home/catalpa/nas/program/text-generation-webui/server.py", line 457, in create_interface
    shared.gradio['interface'].launch(prevent_thread_lock=True, share=shared.args.share, server_name='0.0.0.0', server_port=shared.args.listen_port, inbrowser=shared.args.auto_launch, auth=[tuple(cred.split(':')) for cred in gradio_auth_creds] if gradio_auth_creds else None)
  File "/home/catalpa/anaconda3/envs/textgen/lib/python3.10/site-packages/gradio/blocks.py", line 1381, in launch
    server_name, server_port, local_url, app, server = networking.start_server(
  File "/home/catalpa/anaconda3/envs/textgen/lib/python3.10/site-packages/gradio/networking.py", line 122, in start_server
    raise OSError(
OSError: Port 9441 is in use. If a gradio.Blocks is running on the port, you can close() it or gradio.close_all().

System Info

ubuntu 22, 3080 ti 12G

[oobabooga]

I can confirm this issue. Strangely, it only seems to happen when a custom --listen-port is specified.

[dancingmadkefka]

Happens to me too, load the webui do whatever you want for as long as you want, then if you restart the webui it's gone. To get it back, I have to stop it, reset the interface with netsh int ipv4 reset(I think that was the command) and then I have to start the server again and THEN do the interface proxy, if I do it the other way around it does not work. I use the recommended 7860 port

[catalpaaa]

I can confirm this issue. Strangely, it only seems to happen when a custom --listen-port is specified.

I will go test it with newer gradio and current one but with queue off. I heard blcok.queue() is causing this.

I would just add a conditional sleep 120s before restarting the interface in the loop at the end of server.py for now

[catalpaaa]

I can confirm this issue. Strangely, it only seems to happen when a custom --listen-port is specified.

I will go test it with newer gradio and current one but with queue off. I heard blcok.queue() is causing this.

I would just add a conditional sleep 120s before restarting the interface in the loop at the end of server.py for now

newer version just break all the ui and make everything looks autistic, disabling queue will make canceling an event not possible. a1111's web with gradio 3.16.2 dose not have this issue with queue disable, he might have a diff way to stop a even.

queue() holds something for 60s or more after you do close(), without rewrite a bunch of stuff i would just add a arg that allows user to sleep for say 60s or more before create_interface() after shared.gradio['interface'].close()

[mcmonkey4eva]

That's certainly one way to workaround that. Ideally the root issue should just be fixed - ie gradio should be made to properly close the port somehow. The weird thing is it's able to retain the lock on the port for a period of time even if you entirely close the python process.

[oobabooga]

I agree with @mcmonkey4eva that it would be best to get this solved upstream. I dislike the idea of adding a new command-line argument to work around an upstream bug.

I have created a bug report for gradio here https://github.com/gradio-app/gradio/issues/3675

[tensiondriven]

@oobabooga I think I got to the bottom of this issue, or at least, I have a branch that shows this working/not working, when using queue. It looks like the calls to load() inside the blocks context manager are causing gradio to hold its connection open. I haven't looked into how to change the calls to load() so they can be used in a way that doesn't cause this, but it is a step toward a solution.

Here's a branch that shows an example of a graceful startup/shutdown:

https://github.com/tensiondriven/text-generation-webui/blob/graceful-shutdown-demo/server.py#L21

Sorry about all the automatic code reformatting. I'm using VSCode with the default python linter from Pylance. I don't do a lot of python, so this is new to me. If you can tell me what type of linter I should be using, happy to do so. I also think there's some value in standardizing on something that is common - but i don't know enough about the python ecosystem yet to have an opinion.

Anyhoo, I agree that this is an issue that should be fixed upstream, but i think it may be more of a footgun for all gradio developers, and not something that gradio is likely to fix any time soon.

[oobabooga]

That's a relevant finding, @tensiondriven. I trust that they will solve this issue, the gradio developers are very serious about handling confirmed bugs.

[tensiondriven]

@oobabooga @catalpaaa fwiw, I don't think this is something Gradio will fix in the near term; I think the bug is due to how text-generation-ui is holding onto references to gradio, which are preventing gradio from being able to free memory and close. I saw one other report of this on the gradio github issues, but it wasn't clearly acknowledged as a gradio bug, from what I remember. So, i think users of text-generation-ui will continue to encounter this unless we do some work to decouple gradio from the rest of the app.

I think it should be possible, for example, to start/stop gradio while still keeping a model in memory or keeping some state around. The default/standard implementation of gradio might not make this obvious, which is why I think so many people who use gradio encounter this bug. One should even be able to run text-generation-webui with only the api endpoint running and skip running gradio entirely.

[mcmonkey4eva]

@tensiondriven it was already fixed actually! Gradio 3.28 has the fix, thanks to https://github.com/gradio-app/gradio/pull/3959