[bug]: ConnectionAbortedError: [WinError 10053]

[Yorzaren]

Is there an existing issue for this?

• [X] I have searched the existing issues

OS

Windows

GPU

cuda

VRAM

8GB

What happened?

I'm currently using the 2.2.0 Pre-Release everything seems to be working fine with the images getting generated using the web GUI on firefox. However, I noticed I get an error message in the console every once in a while. The images generate fine still and the web client is still connected.

Traceback (most recent call last):
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\site-packages\eventlet\hubs\selects.py", line 59, in wait
    listeners.get(fileno, hub.noop).cb(fileno)
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\site-packages\eventlet\greenthread.py", line 221, in main
    result = function(*args, **kwargs)
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\site-packages\eventlet\wsgi.py", line 837, in process_request
    proto.__init__(conn_state, self)
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\site-packages\eventlet\wsgi.py", line 352, in __init__
    self.finish()
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\site-packages\eventlet\wsgi.py", line 751, in finish
    BaseHTTPServer.BaseHTTPRequestHandler.finish(self)
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\socketserver.py", line 811, in finish
    self.wfile.close()
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\socket.py", line 723, in write
    return self._sock.send(b)
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\site-packages\eventlet\greenio\base.py", line 401, in send
    return self._send_loop(self.fd.send, data, flags)
  File "C:\Users\User\InvokeAI\installer_files\env\envs\invokeai\lib\site-packages\eventlet\greenio\base.py", line 388, in _send_loop
    return send_method(data, *args)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
Removing descriptor: 3628

Screenshots

No response

Additional context

No response

Contact Details

No response

[Jooju]

I have this same error in 2.2.5-rc1. It occasionally stalls image generation. Sending a Ctrl+Z (^Z) to the terminal allows the process to continue. Each time has a different "removing descriptor" number.

Traceback (most recent call last): File "D:\invokeai\.venv\lib\site-packages\eventlet\hubs\selects.py", line 59, in wait listeners.get(fileno, hub.noop).cb(fileno) File "D:\invokeai\.venv\lib\site-packages\eventlet\greenthread.py", line 221, in main result = function(*args, **kwargs) File "D:\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 837, in process_request proto.__init__(conn_state, self) File "D:\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 352, in __init__ self.finish() File "D:\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 751, in finish BaseHTTPServer.BaseHTTPRequestHandler.finish(self) File "C:\Users\#######\AppData\Local\Programs\Python\Python310\lib\socketserver.py", line 811, in finish self.wfile.close() File "C:\Users\#######\AppData\Local\Programs\Python\Python310\lib\socket.py", line 723, in write return self._sock.send(b) File "D:\invokeai\.venv\lib\site-packages\eventlet\greenio\base.py", line 401, in send return self._send_loop(self.fd.send, data, flags) File "D:\invokeai\.venv\lib\site-packages\eventlet\greenio\base.py", line 388, in _send_loop return send_method(data, *args) ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine Removing descriptor: 3472

Traceback (most recent call last): File "D:\invokeai\.venv\lib\site-packages\eventlet\hubs\selects.py", line 59, in wait listeners.get(fileno, hub.noop).cb(fileno) File "D:\invokeai\.venv\lib\site-packages\eventlet\greenthread.py", line 221, in main result = function(*args, **kwargs) File "D:\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 837, in process_request proto.__init__(conn_state, self) File "D:\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 352, in __init__ self.finish() File "D:\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 751, in finish BaseHTTPServer.BaseHTTPRequestHandler.finish(self) File "C:\Users\#######\AppData\Local\Programs\Python\Python310\lib\socketserver.py", line 811, in finish self.wfile.close() File "C:\Users\#######\AppData\Local\Programs\Python\Python310\lib\socket.py", line 723, in write return self._sock.send(b) File "D:\invokeai\.venv\lib\site-packages\eventlet\greenio\base.py", line 401, in send return self._send_loop(self.fd.send, data, flags) File "D:\invokeai\.venv\lib\site-packages\eventlet\greenio\base.py", line 388, in _send_loop return send_method(data, *args) ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine Removing descriptor: 3516

[ilike1664]

Same issue.

OS Windows

GPU cuda

VRAM 24GB

InvokeAI version 2.2.0

Think I have pinned this issue down to requesting more than 1 image using the default stable-diffusion-1.5 model, doesn't seem to care which sampler is being used. Switching to Openjourney allowed creation of more than 1 image without issue, switch back gave the same error for more than 1 image. Full session log attached.

InvokeAI WinError 10053.txt

[Edit] Managed to re-create the error using the openjourney model but only after asking for it to generate 8 images at a time, it failed on 6/8 and 7/8.

[WillyamBradberry]

It seams like "invoke AI" doesn't like disk D, will try default C...

[bani6809]

Same issue.

OS Windows 10

GPU RTX A4000

VRAM 16GB

InvokeAI version 2.3.0-rc4

Happens any time you request more than 1 image.

Traceback (most recent call last): | 2/50 [00:00<00:07, 6.37it/s] File "D:\InvokeAI.venv\lib\site-packages\eventlet\hubs\selects.py", line 59, in wait listeners.get(fileno, hub.noop).cb(fileno) File "D:\InvokeAI.venv\lib\site-packages\eventlet\greenthread.py", line 221, in main result = function(*args, **kwargs) File "D:\InvokeAI.venv\lib\site-packages\eventlet\wsgi.py", line 837, in process_request proto.init(conn_state, self) File "D:\InvokeAI.venv\lib\site-packages\eventlet\wsgi.py", line 352, in init self.finish() File "D:\InvokeAI.venv\lib\site-packages\eventlet\wsgi.py", line 751, in finish BaseHTTPServer.BaseHTTPRequestHandler.finish(self) File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\socketserver.py", line 811, in finish self.wfile.close() File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\socket.py", line 723, in write return self._sock.send(b) File "D:\InvokeAI.venv\lib\site-packages\eventlet\greenio\base.py", line 401, in send return self._send_loop(self.fd.send, data, flags) File "D:\InvokeAI.venv\lib\site-packages\eventlet\greenio\base.py", line 388, in _send_loop return send_method(data, *args) ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine Removing descriptor: 4576

maybe invokeai isn't properly closing previous connections when opening new ones when Images > 1 ?

[Harvester62]

OS Windows 10 Pro

GPU GTX1070

VRAM 8GB

InvokeAI version 2.3.0

I am getting the same errors here on Windows 10 Pro with InvokeAI v2.3.0 release when I batch generate images (eg. 4 images) , but only with some models selected, like: stable-diffusion-1.5 diffuser, openjourney-v2 ckpt, redshift-diffusion-1 ckpt but not with dreamlike-diffusion-1 ckpt. Despite the errors, the images are generated correctly. Here is just on example (the Removing descriptor number changes every time).

Traceback (most recent call last): File "C:\invokeai_23.venv\lib\site-packages\eventlet\hubs\selects.py", line 59, in wait listeners.get(fileno, hub.noop).cb(fileno) File "C:\invokeai_23.venv\lib\site-packages\eventlet\greenthread.py", line 221, in main result = function(*args, **kwargs) File "C:\invokeai_23.venv\lib\site-packages\eventlet\wsgi.py", line 837, in process_request proto.init(conn_state, self) File "C:\invokeai_23.venv\lib\site-packages\eventlet\wsgi.py", line 352, in init self.finish() File "C:\invokeai_23.venv\lib\site-packages\eventlet\wsgi.py", line 751, in finish BaseHTTPServer.BaseHTTPRequestHandler.finish(self) File "C:\Program Files\Python310\lib\socketserver.py", line 811, in finish self.wfile.close() File "C:\Program Files\Python310\lib\socket.py", line 723, in write return self._sock.send(b) File "C:\invokeai_23.venv\lib\site-packages\eventlet\greenio\base.py", line 401, in send return self._send_loop(self.fd.send, data, flags) File "C:\invokeai_23.venv\lib\site-packages\eventlet\greenio\base.py", line 388, in _send_loop return send_method(data, *args) ConnectionAbortedError: [WinError 10053] Connessione interrotta dal software del computer host Removing descriptor: 4536

[lstein]

@psychedelicious I'm going to try to fix this annoying error. Can I ask whether anyone has been able to reproduce the error using the command-line client? I am betting that it is an issue with the connection between the backend and the web frontend.

[Harvester62]

I did a few tests the the CLI (also my first time I used it) and I generated batches of 4 images with different prompts, and arguments -n4 -s20 -Addim with the stable-diffusion-1.5 model. The above error message didn't display.

[Harvester62]

I did another round of tests to check if the issue might be caused by an anti malware, antivirus or the firewall, disconnecting first from the Internet, then deactivating one at a time the above and generating 4 images every time. Currently running on my notebook on Windows 10 Pro are: Malwarebytes, Symantec Endpoint Protection antivirus client, and Firewall (both driven by SEP and by Windows Defender after SEP deactivation). None of them seemed to be the source of the error because every time the same error message was displayed after the first or any subsequent image generation in the round of 4.

[psychedelicious]

Hmm. Without Windows available to me, I can't really troubleshoot this. @blessedcoolant any ideas for troubleshooting this issue?

[blessedcoolant]

Never faced this but I'll try to troubleshoot this later in the day.

[Harvester62]

In case it helps, I first started the v2.3.0 backend (default option 2), then I executed the developmental version of InvokeAI from the C:\InvokeAI_test\InvokeAI\invokeai\frontend folder with the command yarn dev and get access to InvokeAI at URL http://localhost:5173/ and the above error messages never raise during multiple images generation. Here is a screenshot after two rounds of generation of 4 images.

Immediately after that I opened URL http://localhost:9090 and did the same generation of 4 images getting the errors in the console (the same already opened to run the previous tests) and this is a screenshot:

[blessedcoolant]

So this happens only during production build?

[Harvester62]

Apparently, yes.

[psychedelicious]

Oh boy that's fun. Never would have guessed that dev mode work work better. I wonder if the vite proxy is somehow mitigating the issue

[Harvester62]

Unfortunately, this error is still happening in the latest v2.3.1 even before running any generation process:

It doesn't happen with the latest developmental version (SHA-1: e700da23d82d822cc3d39fa09cf246387f3d9d7f)

[psychedelicious]

I don't trust that the latest dev version really fixes this but that's good to know.

Did a bit of research and have a two ideas to troubleshoot the issue.

Idea 1 - socket.io is trying to send data that is too large

It's possible that a generated image or something in the outputs directory is causing the problem.

To test this, first rename your outputs folder to something else eg _outputs, then make a new empty outputs folder. Does the issue still occur? If it stops the issue from occurring, try restoring the original outputs folder and see if the issue occurs.

Idea 2 - the async library eventlet is the cause

Try rolling back eventlet to before the issue first occurred. So that means to v0.31.1.

First you need to activate the virtual environment or choose option 8 'developer console'.

Then run these commands:

pip uninstall eventlet
pip install eventlet==0.31.1

Finally run InvokeAI and see if the issue still occurs.

[Harvester62]

OK, here is my fist test on your "Idea 1". Since I was pointing the output directory somewhere else, I simply restored the usage of the standard output directory (C:\invokeai_23\outputs) which was emply (empty folders plus a .invoke_history file, and run a 4 images generation. After the third generation the error was triggered. It seems not to be due to something inside the output directory, unless it's the hidden .invoke_history file, so I deleted that file and ran three batches of 4 generation each, and the error was triggered after the second and third generation in the first batch, none in the second, and after the fourth generation in the third batch.

Perhaps it was a useless test but I thought it might be good to knwow it. Now, I would "try" to test your "Idea 2" option and hope to do not disrupt my local installation of InvokeAI. Before proceeding I reinstated/recreated the .invoke_history file inside the \output directory.

The following is the console after unistall and then reinstall of eventlet v0.31.1

Unfortunately, this prevented the running of the Web UI as per the console errors in the following screenshot:

To restore the system in its previous state I did uninstall eventlet v0.31.1 and reinstalled v0.33.3, then because of the error/warning about dnspython I uninstalled the v1.16.0 and reinstalled v2.2.1. InvokeAI was then running fine. Sorry, I wasn't able to test your second idea, but if you might provide more instructions to prevent the errors the changes raised, I can give it another try.

[Void2258]

Yes, my report (#3027) seems to be the same error.

EDIT: Error persists in 2.3.3

[Void2258]

This error appears to be proliferating. I am now seeing it repeat a bunch of times on startup with 2.3.3. Still doesn't appear to actually do anything, but it's making the console almost unreadable as it swamps all other messages.

[scpedicini]

Confirmed - I'm hardwired in to a local LAN and when using Chrome on my laptop to access the LAN Win 10 Desktop running newest version 2.3.3 it happens during each subsequent batch image (once every image), and occasionally if I click on thumbnails of images.

Traceback (most recent call last):
  File "C:\dev\invokeai\.venv\lib\site-packages\eventlet\hubs\selects.py", line 59, in wait
    listeners.get(fileno, hub.noop).cb(fileno)
  File "C:\dev\invokeai\.venv\lib\site-packages\eventlet\greenthread.py", line 221, in main
    result = function(*args, **kwargs)
  File "C:\dev\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 837, in process_request
    proto.__init__(conn_state, self)
  File "C:\dev\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 352, in __init__
    self.finish()
  File "C:\dev\invokeai\.venv\lib\site-packages\eventlet\wsgi.py", line 751, in finish
    BaseHTTPServer.BaseHTTPRequestHandler.finish(self)
  File "C:\Python\Python310-x64\lib\socketserver.py", line 811, in finish
    self.wfile.close()
  File "C:\Python\Python310-x64\lib\socket.py", line 723, in write
    return self._sock.send(b)
  File "C:\dev\invokeai\.venv\lib\site-packages\eventlet\greenio\base.py", line 401, in send
    return self._send_loop(self.fd.send, data, flags)
  File "C:\dev\invokeai\.venv\lib\site-packages\eventlet\greenio\base.py", line 388, in _send_loop
    return send_method(data, *args)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
Removing descriptor: 4068

This was using the seek-art-mega model with a batch count of 16 images. Restarting the invokeai server did not seem to make a difference.