"--listen" not working (?!)

[Malrama]

Describe the bug

So, I am not 100% sure what "--listen" is supposed to do, but I assume it makes my client visible for other browsers in my network (or even on the internet with port forwarding)?

So I used "--listen" and everything is setup 0.0.0.0:7860. So dar so good. Localhost:7860 is working perfectly.

However, I can not access Text-generation-webui either within my homenetwork or outside my homenetwork with my IP-Adresse.

Neither 192.XXX[...]:7860 (Homenetwork) nor 77:XXX[...]:7860 (Internet) is working when I try to connect from my phone. All ports are free and setup. Am I missing something? Maybe I don't quite understand what "--listen" does but I hope and assume that it let's me run this software from different devices/networks? Like "--web" in InvokeAI?

I am not a big fan of "--share" because that involves a (not mine) webserver inbetween. Thank you and greetings

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Run with "--listen" and then try to connect from a different device to your Network-Adress or Internet-IP adress.

Screenshot

Run with "--listen" and then try to connect from a different device to your Homenetwork-Adressor Internet-IP adress.

Logs

Nothing bad

System Info

All up-to-date
12GB Nvidia RTX 3060 OC
Ryzen 7 3700X
Everything working good!

[dblacknc]

It has worked well for me all along, including today after a git pull. OS is Ubuntu 22.04.

A firewall could block access from the LAN, while allowing localhost to work.

[Malrama]

It has worked well for me all along, including today after a git pull. OS is Ubuntu 22.04.

A firewall could block access from the LAN, while allowing localhost to work.

Weird. I have port 7860 open in my firewall and my Router is forwading the port to my PC. I have the same setup as in InvokeAI and it works there but not for this software. Not locally, not on the internet. Am I missing something at this point? Am I crazy?

[dblacknc]

On either Windows or Linux, you could try "netstat -a -n" to verify what address a socket is bound to. I'm using --listen --listen-port 80. For your netstat -a -n output, you'd look for :7860. On Linux "netstat -an | fgrep :80" returns for the first line:

tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN

When the UI starts, this is also displayed towards the end of the output, confirming use of the wildcard address.

Running on local URL: http://0.0.0.0:80

If you see 0.0.0.0 as above, the app is listening on all host interfaces, meaning there's some other reason you can't reach it. The most likely reasons remain either it's not properly forwarded - since you mention forwarding, or a firewall rule on your router or host is blocking it. E.g. if I'm forwarding from the Internet on my AT&T router, since its firewall is enabled I have to create both a firewall rule allowing the port, as well as a separate forwarding rule.

[VRArt1]

I'm also having this issue. I'm using the following command arguments: --auto-devices --chat --wbits 4 --groupsize 128 --listen --listen-port 6001 --model vicuna-13b-GPTQ-4bit-128g --model-menu --extensions sd_api_pictures --auto-launch

[azumukupoe]

https://github.com/oobabooga/text-generation-webui/blob/main/docs/WSL-installation-guide.md#bonus-port-forwarding

[Kallamamran]

I have the same problem. A few weeks ago I followed the manual install and that installation ran --listen and --listen-port 7840 without any problem. With the one-click-install it doesn't work (just tried)

I need to be able to set the port since I have other stuff on port 7860 😥

[Kallamamran]

Found a solution!

Edit the INITIAL_PORT_VALUE in the file .\installer_files\env\Lib\site-packages\gradio\networking.py with the port-number you want to use!

[Kallamamran]

Would be nicer if the start ARGS worked though since "--extensions api" and "--auto-devices" doesn't work either :/

[bbecausereasonss]

I have had a ton of issues being able to open WSL2/Ubuntu gradio interface in windows. Sometimes it worked sometimes it does not, and now it has stopped for no apparent reason. Nothing I do including port forwarding has worked :/

[Zyadon7]

https://github.com/oobabooga/text-generation-webui/blob/main/docs/WSL-installation-guide.md#bonus-port-forwarding

This worked for me. I'm hosting on WSL Ubuntu through Win10. Thank you.

[pinballelectronica]

Found a solution!

Edit the INITIAL_PORT_VALUE in the file .\installer_files\env\Lib\site-packages\gradio\networking.py with the port-number you want to use!

Line 27- In general using 127.0.0.1 in code is not a good idea. It doesn't port well, including to Cloud and other OS environs. Better to use 0.0.0.0

[pvutov]

I've had no issue connecting from my phone to 192.x.y.z:7860 if I start with --listen.

Where I stumble is in exposing the whole thing to the web. --share works, but I want to host it on my own domain. --listen-host and --listen-port seem intended for this, but not sufficient. At least in my case (my PC <-> ssh tunnel through the router NAT <-> public-facing reverse proxy server <-> visitor; I know that this tunnel mechanism works in principle because I use it to expose other things successfully).

I use the command: python server.py --model ggml-vic7b-q5_0.bin --load-in-8bit --listen-host https://somedomain.com/chat --listen-port 7860

With that command, localhost:7860 works. somedomain.com/chat does not work - it reaches the web UI server, but the UI is never rendered. A lot of HTML does get served but the javascript magic seems to fail to find its resources or something.

Interestingly it gets a bit further with --share. In that case, the autogenerated link works (of course). somedomain.com/chat still does not work, but it gets as far as displaying the 'Loading' text (and not progressing past that). Without --share, it's just a blank page.

If I add --listen, the server throws a bunch of python exceptions and fails to start.

python server.py --model ggml-vic7b-q5_0.bin --load-in-8bit --listen --listen-host https://somedomain.com/chat --listen-port 7860

[...]
File "/home/username/.conda/envs/textgen/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
[...]
File "/home/username/repos/text-generation-webui/modules/logging_colors.py", line 96, in new
    args[1].msg = color + args[1].msg + '\x1b[0m'  # normal
TypeError: can only concatenate str (not "gaierror") to str

[pvutov]

I simplified my setup by getting rid of /chat in my nginx forwarding config, and it works. E.g.

python server.py --model ggml-vic7b-q5_0.bin --load-in-8bit --listen-host https://somedomain.com/chat --listen-port 7860

Loads perfectly fine at https://somedomain.com - even when the /chat part is left in the command. So that part seems to be ignored or at least makes no difference for now.

Nginx config has:

location / {
        proxy_pass http://127.0.0.1:7860/;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

The issue described previously only happens if location /chat is used instead.

[kierenalt255]

I simplified my setup by getting rid of /chat in my nginx forwarding config, and it works. E.g.

python server.py --model ggml-vic7b-q5_0.bin --load-in-8bit --listen-host https://somedomain.com/chat --listen-port 7860

Loads perfectly fine at https://somedomain.com - even when the /chat part is left in the command. So that part seems to be ignored or at least makes no difference for now.

Nginx config has:

location / {
        proxy_pass http://127.0.0.1:7860/;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

The issue described previously only happens if location /chat is used instead.

Hi @pvutov, how did you get around the WebSocket connection issue ? index.js:462 WebSocket connection to 'ws://somedomain.com:81/queue/join' failed:

[pvutov]

I don't recognize that error, but it's been a while..

[ashleykleynhans]

Hi @pvutov, how did you get around the WebSocket connection issue ? index.js:462 WebSocket connection to 'ws://somedomain.com:81/queue/join' failed:

You can resolve it like this:

server {
        listen 7860;

        location /ws {
            proxy_http_version 1.1;
            proxy_set_header Accept-Encoding gzip;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            add_header Cache-Control no-cache;
            proxy_set_header Host $host;

            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
	        proxy_set_header   X-Real-IP		$remote_addr;
            proxy_pass http://localhost:3001;
        }

        location /queue/join {
            proxy_http_version 1.1;
            proxy_set_header Accept-Encoding gzip;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            add_header Cache-Control no-cache;
            proxy_set_header Host $host;

            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
	        proxy_set_header   X-Real-IP		$remote_addr;
            proxy_pass http://localhost:3001;
        }

        location / {
            add_header Cache-Control no-cache;
            proxy_pass http://localhost:3001;
            proxy_http_version 1.1;
            proxy_set_header Accept-Encoding gzip;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Host $host;

            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_intercept_errors on;
            error_page 502 =200 @502;
        }
    }

[ChrissoChrisso]

I'm having the same issue. I've used the 1 click installer in Linux Mint 21.2 (which is based on Ubuntu 22.04). It's a fresh OS install + updates + nvidia drivers + build-essential + openssh-server + oobabooga.

The oobabooga web interface can be accessed from the machine running it, but not from other machines on the LAN. I want to be able to reach the oobabooga web interface from other machines on my LAN too.

On the Machine running oobabooga:

1. The firewall is disabled:

~$ sudo ufw status
Status: inactive

2. An nmap scan of localhost reveals that the machine is listening on port 22 for ssh and 7860 for oobabooga as expected:

~$ nmap localhost -p 22,7860
Starting Nmap 7.80 ( https://nmap.org ) at 2023-08-20 00:12 CDT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000027s latency).

PORT     STATE SERVICE
22/tcp   open  ssh
7860/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 0.02 seconds

3. Netstat shows this output:

~$ netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 localhost:7950          0.0.0.0:*               LISTEN     
tcp        0      0 localhost:7860          0.0.0.0:*               LISTEN     
tcp        0      0 localhost:ipp           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:ssh             0.0.0.0:*               LISTEN     
tcp        0      0 localhost:domain        0.0.0.0:*               LISTEN

4. I've tried starting oobabooga with the --listen flag, and even with sudo privileges + the --listen flag and those didn't make oobabooga reachable from the LAN:

~/Desktop/oobabooga_linux$ ./start_linux.sh --listen
2023-08-19 23:23:36 INFO:Loading the extension "gallery"...
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

From another machine on the LAN, an nmap scan of the oobabooga machine shows that p 22 is reachable but 7860 isn't:

~$ nmap 192.168.11.220 -p 22,7860
Starting Nmap 7.93 ( https://nmap.org ) at 2023-08-19 23:53 CDT
Nmap scan report for 192.168.11.220
Host is up (0.00020s latency).

PORT     STATE  SERVICE
22/tcp   open   ssh
7860/tcp closed unknown

Nmap done: 1 IP address (1 host up) scanned in 0.07 seconds

I do not want my oobabooga instance to be reachable from the internet (so I don't want to choose the public link option). I just want it to be reachable on my LAN. What do I need to change to make oobabooga's webpage accessible from my LAN? I suspect that elevated privilege or permission may be needed to make oobabooga listen to p7860 on a network interface? I just don't know what to change where to make that happen.

Thanks!!

[mongolu]

I don't think this is the propper way of passing args to webui "./start_linux.sh --listen"

Instead, do it by setting an env var OOBABOOGA_FLAGS or writing them in the CMD_FLAGS.txt. You can read it here

[ChrissoChrisso]

@mongolu thank you so much! Writing the args in the CMD_FLAGS.txt was the fix for me :+1:

I added --listen followed by a carriage return to the /oobabooga/CMD_FLAGS.txt file, and now can reach oobabooga from the LAN after starting it normally. The full contents of my working CMD_FLAGS.txt file are shown here:

~/Desktop/oobabooga_linux$ cat CMD_FLAGS.txt 
--listen

Notably after passing the --listen flag in the CMD_FLAGS.txt file, oobabooga runs listening to 0.0.0.0 instead of 127.0.0.1:

~/Desktop/oobabooga_linux$ ./start_linux.sh
2023-08-20 06:56:46 INFO:Loading the extension "gallery"...
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.

[github-actions[bot]]

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.

[nervousapps]

@mongolu thank you so much! Writing the args in the CMD_FLAGS.txt was the fix for me 👍

I added --listen followed by a carriage return to the /oobabooga/CMD_FLAGS.txt file, and now can reach oobabooga from the LAN after starting it normally. The full contents of my working CMD_FLAGS.txt file are shown here:

~/Desktop/oobabooga_linux$ cat CMD_FLAGS.txt 
--listen

Notably after passing the --listen flag in the CMD_FLAGS.txt file, oobabooga runs listening to 0.0.0.0 instead of 127.0.0.1:

~/Desktop/oobabooga_linux$ ./start_linux.sh
2023-08-20 06:56:46 INFO:Loading the extension "gallery"...
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.

It must be done also for the docker install