email-oauth2-proxy
email-oauth2-proxy copied to clipboard
simonrob
Server disconnected when SMTP proxy on another computer
EDIT: When I disabled the signature then message was sent as it should.
So the problem is not because the proxy is on another computer - the problem must be with parsing the signature HTML which looks like:
<html>
<body>
<font size=1 color="#0000FF"><br>
</font><img src="file://C:\imgs\signature.gif" width=18 height=18 alt="sig"><font face="Comic Sans MS" color="#0000FF"> <b><i>My name</i></b></font> <img src="file://C:\imgs\sig.gif" width=18 height=18 alt="sig"><br></body>
</html>
or maybe with sending the .gifs which are part of the signature.
END OF EDIT
Hello again,
in the meantime while still testing running the proxy on Windows XP I've made a temporary setup where the proxy runs on Windows 10 computer which is in the same local network as the Windows XP computer running Eudora email client and has static IP.
I didn't expect problems but while receiving email via POP + gmail works well, sending via SMTP + gmail never gets to the end when the proxy is on another computer - the progress bar in Eudora email client always stops at almost the end and after a while there is an error:
Error reading from network
Cause: connection aborted due to timeout or other failure (10053)
I saw issue #36 even before setting up the proxy on another computer and I did set local_address for both POP and SMTP and the log looks right until communication just stops.
I've compared the logs from the working case where SMTP server is on the same computer as email client with the logs where the proxy is on another computer and everything looks the same except in the case where the proxy is on another computer the server (gmail) suddenly disconnects. Here is how the logs look like at the point where disconnect happens:
.
.
.
2022-08-15 23:26:14,505: SMTP (192.168.1.102:1465; 192.168.1.100:3838->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) --> b'RCPT TO:<[[ email address removed ]]@yahoo.com>\r\n'
2022-08-15 23:26:14,557: SMTP (192.168.1.102:1465; 192.168.1.100:3838->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) <-- b'250 2.1.5 OK [[ some numbers removed ]] - gsmtp\r\n'
2022-08-15 23:26:14,559: SMTP (192.168.1.102:1465; 192.168.1.100:3838->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) --> b'DATA\r\n'
2022-08-15 23:26:14,664: SMTP (192.168.1.102:1465; 192.168.1.100:3838->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) <-- b'354 Go ahead [[ some numbers removed ]] - gsmtp\r\n'
2022-08-15 23:26:14,666: SMTP (192.168.1.102:1465; 192.168.1.100:3838->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) --> b'X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9\r\n'
2022-08-15 23:36:14,716: SMTP (192.168.1.102:1465; 192.168.1.100:3838->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) <-- [ Server disconnected ]
192.168.1.100 is Windows XP computer with Eudora email client 192.168.1.102 is Windows 10 computer running Email OAuth2 Proxy
I configured the proxy with:
[SMTP-1465]
local_address = 192.168.1.102
server_address = smtp.gmail.com
server_port = 465
[POP-1996]
local_address = 192.168.1.102
server_address = pop.gmail.com
server_port = 995
I am using the most recent code and have tried to disable the firewalls but still the same.
The last line Eudora sends is always:
2022-08-15 23:26:14,666: SMTP (192.168.1.102:1465; 192.168.1.100:3838->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) --> b'X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9\r\n'
while in the working configuraton (where the proxy and the email client are on the same computer) these lines look like:
2022-08-15 23:34:27,787: SMTP (localhost:1465; 127.0.0.1:49213->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) --> b'X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9\r\n'
2022-08-15 23:34:27,803: SMTP (localhost:1465; 127.0.0.1:49213->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) --> b'Date: Mon, 15 Aug 2022 23:34:21 +0200\r\nTo: [[ email address removed ]]@yahoo.com\r\nFrom: "Name" <[[ email address removed ]]@gmail.com>\r\nSubject: Some subject\r\nMime-Version: 1.0\r\nContent-Type: text/plain; charset="us-ascii"; format=flowed\r\n\r\nMessage body\r\n\r\n.\r\n'
2022-08-15 23:34:28,412: SMTP (localhost:1465; 127.0.0.1:49213->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) <-- b'250 2.0.0 OK [[ some numbers removed ]] - gsmtp\r\n'
2022-08-15 23:34:28,428: SMTP (localhost:1465; 127.0.0.1:49213->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) --> b'QUIT\r\n'
2022-08-15 23:34:28,475: SMTP (localhost:1465; 127.0.0.1:49213->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) <-- b'221 2.0.0 closing connection [[ some numbers removed ]] - gsmtp\r\n'
2022-08-15 23:34:28,475: SMTP (localhost:1465; 127.0.0.1:49213->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) <-- [ Server disconnected ]
This is what is logged by Eudora when trying to send an email:
MAIN 16: 1.44 Preparing messages to Send: 1
MAIN 16: 1.55 RSET
MAIN 16: 1.55 MAIL FROM:<[email protected]>
MAIN 16: 1.55 RCPT TO:<[email protected]>
MAIN 16: 1.55 DATA
MAIN 16: 1.55 Test
MAIN 16: 1.55 Sending file "C:\imgs\signature.gif"...
MAIN 16: 1.55 Test
MAIN 16: 1.55 Preparing messages to Send: 0
4776 16: 1.55 Open 192.168.1.102:1465
4776 8: 2.34 Dialog: "Error reading from network\r\n"
4776 8: 2.34 Dialog: "\r\n"
4776 8: 2.34 Dialog: "Cause: connection aborted due to timeout or other failure (10053)"
And this is what is logged by Eudora in working configuration where the proxy and the email client are on the same computer:
MAIN 16: 0.00 Preparing messages to Send: 1
MAIN 16: 0.00 RSET
MAIN 16: 0.00 MAIL FROM:<[email protected]>
MAIN 16: 0.00 RCPT TO:<[email protected]>
MAIN 16: 0.00 DATA
MAIN 16: 0.00 Message subject
MAIN 16: 0.00 Preparing messages to Send: 0
1088 16: 0.00 Open 127.0.0.1:1465
Thanks for this detailed report.
I would be quite surprised if this is related to the email signature because the proxy does no further editing or parsing of the connection once it has handled authentication. In addition, if an error was encountered by the proxy it would print a message to the log before closing the connection. Still, given that disabling the signature seems to fix email sending, I'm happy to look into this a little further.
Can you send successfully if you use both the proxy and email client (with a signature) on the Windows 10 computer? If so, is there anything that could be filtering network connections between the two computers?
I did a few more tests and here are the results:
Eudora on one computer and OAuth2 Proxy on another computer:
- Plain text signature --> works OK
- Signature containing HTML --> works OK
- Signature contains .gif image --> not working
Eudora and OAuth2 Proxy on the same computer (in Windows 7 virtual machine inside Windows 10 computer):
- Everything works OK (even when the signature contains .gif image)
I copied the signature file from the Eudora I am using for years to the test environment in virtual machine and copied the .gif file to the appropriate directory and everything worked as it should. The problem is only when OAuth2 Proxy is not on the same computer and signature contains .gif image.
There is one more difference between using OAuth2 Proxy on the same computer as the email client vs when both OAuth2 Proxy and the email client are on the same computer - when OAuth2 Proxy is on another computer the authorisation pop up never shows. Authorisation does work well but the tray icon doesn't show the notification while in the case when both OAuth2 Proxy and email client are on the same computer authorisation pop up always shows.
I didn't change anything besides POP and SMPT server configuration at the computer running Eudora since it stopped working on the 7th of June. I tried to send an email with the signature contatining .gif image after disabling antivirus and firewall too but that didn't help either.
This is really interesting. I suspect the double forward slashes or backslash in .gif's URL:
<img src="file://C:\imgs\signature.gif"...
might somehow cause some parsing or encoding problems.
Just now when writing the message I got an idea to test if sending attachments would work and it didn't. I attached a 150 kB .png image and tried to send email without .gif in the signature and the problem was the same as when trying to send the email with .gif in the signature - the only difference is the progress bar now stops at the beginning.
Eudora has an option to choose MIME, BinHex or Uuencode encoding methods, I tried all but the problem is still the same.
Seems as when OAuth2 Proxy is not on the same computer as the email client then I can't send anything besides plain text or HTML. But as soon as there is to send something that has to be encoded then SMTP breaks.
There is one more option in Eudora under Threading - Maximum number of concurent tasks (default 10). I tried to change that to 1 but I still couldn't send an email with .gif in the signature or an email containing attachment when OAuth2 Proxy is on another computer.
EDIT: After some more testing OAuth2 Proxy running in Windows 10 computer froze and I couldn't stop it using Quit command from the tray icon so I had to use Task Manager to end the task and start it again.
I've captured the traffic to and from the Windows 10 computer running OAuth2 Proxy from the computer running email client while sending email with the signature containing .gif images.
Wireshark
Capture filter: host 192.168.1.102
Here is the screenhot where can be seen there is Duplicate ACK in the frame 36 and then there are multiple transmissions of the frame 32. Frame 36 is marked with black by the Wireshark too but in the screenshot it is blue because it is selected.
Duplicate ACK might happen because there was packet loss or the packets arrived out of order but I still couldn't determine what exactly was the cause and/or why that happens.
I captured the traffic several times and the length of the packet that is retransmitted is always 1514 bytes (the problem is always at exactly the same spot) and the last packet received by the proxy is packet number 31 which is in the form of:
2022-08-15 23:26:14,666: SMTP (192.168.1.102:1465; 192.168.1.100:3838->smtp.gmail.com:465; [[ email address removed ]]@gmail.com) --> b'X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9\r\n'
which corresponds to the logs from my first message.
I tested the same proxy running on Windows 10 machine with Eudora running in virtual machine installed on the same Windows 10 computer which is similar to running the proxy and email client on different computers and in that case everything worked well.
Network communication between Windows XP and Windows 10 computers works well and shared folders are accessible without problems.
I will continue with testing in the evening, the next test will be to check if the problem is caused by the length of the packet and not by it's contents - I will test with plain text message longer than 1500 bytes.
After hours of testing I've noticed the main difference between the working configuration and the configuration where sending emails containing signatures with images or attachments doesn't work.
I examined the packets captured with Wireshark and noticed the maximum length of the packets on PC where sending attachments doesn't work is 1514 bytes (payload length 1460 bytes) but the length of the packets in the working configuration is never more than about 130 bytes.
However, the length of the packets is not a problem - I tried to send an email containing only plain text longer than 1500 bytes and everything worked well despite there were packets with length of 1514 bytes.
I spent a few hours more examining some other aspects of the payload (normal and alternative boundary, network MTU and similar) but that wasn't a problem and then I noticed there are some packets with the length of only 2 bytes in the working configuration and after some more investigation I saw the 0x0d 0x0a sequence in the working configuration's packets never occurs twice.
So the problem is TCP packets containing more than one 0x0d 0x0a sequence are never received by OAuth2 Proxy.
I am not sure what decides if TCP packets would or would not be split at CRLF but since Eudora in both configurations is exactly the same then behaviour probably depends on OS and/or network drivers and/or hardware used in communication.
Now the main question is why exactly TCP packets with multiple CRLF sequences are not recognized by the proxy. Is there maybe some flag in the TCP library used by Oauth2 Proxy to allow multiple CRLFs in the single TCP packet?
My next test (probably in the evening) will be to capture the traffic at the Windows 10 computer running the OAuth2 Proxy to see if those packets arrive cut at the first occurrence of CRLF or they arrive in their full length.
Thanks for this very detailed investigation!
Single b'\r\n' messages are indeed important; however, after authentication the proxy should just pass through all messages unedited, so I still think it's fairly unlikely this is caused by the proxy itself (though I'm happy to be proved wrong of course). I suspect you are correct about OS/network issues being the cause.
The only place I can see for the proxy to be causing this is if somehow the checks for empty messages (here and here) are discarding b'\r\n' as empty. However, since this does not happen when running the proxy on the same computer I'm doubtful about this.
This time I started Wireshark capture on both computers to compare the traffic and these are the results:
- The problematic packet (the one that is retransmitted several times) doesn't arrive at the computer running OAuth2 Proxy at all.
- The packet just before, the last packet during the transmission, which is ACK packet sent by the proxy has length of 54 bytes when sent but it has length of 60 bytes on arrival.
This is capture of the last successful packet captured from the computer running OAuth2 Proxy:
Frame 31: 54 bytes on wire (432 bits), 54 bytes captured (432 bits)
0000 00 01 29 d7 1c aa f8 28 19 bd e2 9d 08 00 45 00 ..)....(......E.
0010 00 28 bb 33 40 00 80 06 bb 81 c0 a8 01 66 c0 a8 .([email protected]..
0020 01 64 05 b9 0b c6 ad a2 56 40 8b ab e7 2a 50 10 .d......V@...*P.
0030 fa 1f a9 61 00 00 ...a..
and this is the very same packet when it arrives at the computer running email client:
Frame 38: 60 bytes on wire (480 bits), 60 bytes captured (480 bits)
0000 00 01 29 d7 1c aa f8 28 19 bd e2 9d 08 00 45 00 ..)....(......E.
0010 00 28 bb 33 40 00 80 06 bb 81 c0 a8 01 66 c0 a8 .([email protected]..
0020 01 64 05 b9 0b c6 ad a2 56 40 8b ab e7 2a 50 10 .d......V@...*P.
0030 fa 1f a9 61 00 00 00 00 00 00 00 00 ...a........
Then I tested sending an email containing only plain text without attachment and signature but this time I inserted a few CRLFs in a row to see if more than one 0x0d 0x0a sequence in the same packet would be a problem and sending worked without an error despite there was 0x0d 0x0a 0x0d 0x0a 0x0d 0x0a 0x0d 0x0a in the same packet.
So you are right, more than one CRLF in the same packet isn't a problem. The length of the packets is not a problem either. Furthermore, seems as even ACK packet sent as 54 bytes and received as 60 bytes isn't a problem too because the same happens even when sending works well.
At the moment I can't notice anything that differentiates the packets that do arrive at the computer running proxy from the packet that never arrives :-/ If that isn't a mystery :-)
The email client clearly does send all the packets correctly but in the case some packet contains the attached file that packet never arrives at its destination - even if the attached file is just a text file containing only one letter.
For example, this is the exact packet that never arrives at the destination. It is from the email containing the text file test.txt with just one letter a as an attachment:
0000 44 61 74 65 3a 20 54 68 75 2c 20 31 38 20 41 75 Date: Thu, 18 Au
0010 67 20 32 30 32 32 20 30 33 3a 31 35 3a 31 39 20 g 2022 03:15:19
0020 2b 30 32 30 30 0d 0a 54 6f 3a 20 xx xx xx xx xx +0200..To: xxxxx
0030 xx xx xx xx 40 79 61 68 6f 6f 2e 63 6f 6d 0d 0a [email protected]..
0040 46 72 6f 6d 3a 20 xx xx xx xx xx xx xx xx xx xx From: xxxxxxxxxx
0050 xx xx xx xx xx xx xx xx 40 67 6d 61 69 6c 2e 63 [email protected]
0060 6f 6d 3e 0d 0a 53 75 62 6a 65 63 74 3a 20 54 65 om>..Subject: Te
0070 73 74 20 28 61 74 74 61 63 68 6d 65 6e 74 29 0d st (attachment).
0080 0a 4d 69 6d 65 2d 56 65 72 73 69 6f 6e 3a 20 31 .Mime-Version: 1
0090 2e 30 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 .0..Content-Type
00a0 3a 20 6d 75 6c 74 69 70 61 72 74 2f 6d 69 78 65 : multipart/mixe
00b0 64 3b 0d 0a 09 62 6f 75 6e 64 61 72 79 3d 22 3d d;...boundary="=
00c0 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d ================
00d0 3d 3d 3d 3d 5f 33 37 30 30 33 37 37 38 31 3d 3d ====_370037781==
00e0 5f 22 0d 0a 0d 0a 2d 2d 3d 3d 3d 3d 3d 3d 3d 3d _"....--========
00f0 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 5f 33 37 =============_37
0100 30 30 33 37 37 38 31 3d 3d 5f 0d 0a 43 6f 6e 74 0037781==_..Cont
0110 65 6e 74 2d 54 79 70 65 3a 20 6d 75 6c 74 69 70 ent-Type: multip
0120 61 72 74 2f 61 6c 74 65 72 6e 61 74 69 76 65 3b art/alternative;
0130 0d 0a 09 62 6f 75 6e 64 61 72 79 3d 22 3d 3d 3d ...boundary="===
0140 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d ================
0150 3d 3d 5f 33 37 30 30 33 37 37 38 31 3d 3d 2e 41 ==_370037781==.A
0160 4c 54 22 0d 0a 0d 0a 2d 2d 3d 3d 3d 3d 3d 3d 3d LT"....--=======
0170 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 5f 33 ==============_3
0180 37 30 30 33 37 37 38 31 3d 3d 2e 41 4c 54 0d 0a 70037781==.ALT..
0190 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
01a0 78 74 2f 70 6c 61 69 6e 3b 20 63 68 61 72 73 65 xt/plain; charse
01b0 74 3d 22 75 73 2d 61 73 63 69 69 22 3b 20 66 6f t="us-ascii"; fo
01c0 72 6d 61 74 3d 66 6c 6f 77 65 64 0d 0a 0d 0a 54 rmat=flowed....T
01d0 68 69 73 20 69 73 20 6d 65 73 73 61 67 65 20 62 his is message b
01e0 6f 64 79 2e 0d 0a 0d 0a 2d 2d 3d 3d 3d 3d 3d 3d ody.....--======
01f0 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 5f ===============_
0200 33 37 30 30 33 37 37 38 31 3d 3d 2e 41 4c 54 0d 370037781==.ALT.
0210 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 .Content-Type: t
0220 65 78 74 2f 68 74 6d 6c 3b 20 63 68 61 72 73 65 ext/html; charse
0230 74 3d 22 75 73 2d 61 73 63 69 69 22 0d 0a 0d 0a t="us-ascii"....
0240 3c 68 74 6d 6c 3e 0d 0a 3c 62 6f 64 79 3e 0d 0a <html>..<body>..
0250 3c 66 6f 6e 74 20 73 69 7a 65 3d 33 3e 54 68 69 <font size=3>Thi
0260 73 20 69 73 20 6d 65 73 73 61 67 65 20 62 6f 64 s is message bod
0270 79 2e 3c 62 72 3e 0d 0a 3c 2f 66 6f 6e 74 3e 3c y.<br>..</font><
0280 2f 62 6f 64 79 3e 0d 0a 3c 2f 68 74 6d 6c 3e 0d /body>..</html>.
0290 0a 0d 0a 2d 2d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d ...--===========
02a0 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 5f 33 37 30 30 33 ==========_37003
02b0 37 37 38 31 3d 3d 2e 41 4c 54 2d 2d 0d 0a 0d 0a 7781==.ALT--....
02c0 2d 2d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d --==============
02d0 3d 3d 3d 3d 3d 3d 3d 5f 33 37 30 30 33 37 37 38 =======_37003778
02e0 31 3d 3d 5f 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 1==_..Content-Ty
02f0 70 65 3a 20 74 65 78 74 2f 70 6c 61 69 6e 3b 20 pe: text/plain;
0300 63 68 61 72 73 65 74 3d 22 75 73 2d 61 73 63 69 charset="us-asci
0310 69 22 0d 0a 43 6f 6e 74 65 6e 74 2d 44 69 73 70 i"..Content-Disp
0320 6f 73 69 74 69 6f 6e 3a 20 61 74 74 61 63 68 6d osition: attachm
0330 65 6e 74 3b 20 66 69 6c 65 6e 61 6d 65 3d 22 74 ent; filename="t
0340 65 73 74 2e 74 78 74 22 0d 0a 0d 0a 61 0d 0a 2d est.txt"....a..-
0350 2d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d -===============
0360 3d 3d 3d 3d 3d 3d 5f 33 37 30 30 33 37 37 38 31 ======_370037781
0370 3d 3d 5f 2d 2d 0d 0a 0d 0a 2e 0d 0a ==_--.......
There must be something in this packet because of what that very packet is different from other packets that do arrive at the destination but I just don't see what that could be :-/
Today I started the proxy and when checking email (which works well) the email client couldn't complete the check. Eudora has task bar displaying the status when checking email which looks something like:
Connecting...
Connection established.
...
...
PASS
EHLO
...
...
Fetching emails... (number)
...
QUIT
When checking email for the first time the task bar status stalled at PASS. Then I stopped checking using the STOP command in the email client and when I tried to check email for the 2nd time the status sstalled at Connection established. Then I stopped checking again and when I tried to check email for the 3rd time the status stalled at Connecting....
All that time there wasn't any output in the terminal window from where I started OAuth2 Proxy and seems as each time I was checking email the process stalled one stage earlier than before.
When I tried to exit from OAuth2 Proxy using Quit Email OAuth 2.0 Proxy from the tray icon nothing happened and I could stop the proxy only by using Task Manager. Before stopping the proxy I captured the traffic from the Wireshark in Windows 10 computer running the proxy and TCP packets sent by email client didn't arrive until I stopped the proxy using task manager and started it again.
Seems as there is some connection between "TCP packets not arriving at the destination" and OAuth2 proxy because Wireshark wasn't capturing any traffic from the email client during the time when OAuth2 Proxy was frozen but as soon as OAuth2 Proxy was restarted email checking started to work again and Wireshark could again capture the traffic sent from the email client meaning OAuth2 Proxy was somehow blocking the TCP packets.
It looks to me as:
- OAuth2 Proxy in certain conditions can somehow cause "TCP packets are not delivered" state.
- When "TCP packets are not delivered" state happens during POP access then OAuth2 Proxy freezes and the only way to use it again is to force End Task and to restart the proxy.
- When "TCP packets are not delivered" state happens during SMTP access then OAuth2 Proxy doesn't freeze and it could still be used by stopping the process of sending email(s) from the email client or by waiting for timeout.
The problem is how to identify what conditions exactly are causing "TCP packets are not delivered" state.
Thanks again for the very detailed investigation here!
I wonder whether this is related to operating system sleeping/suspending? There was a similar issue on macOS in an early version of the proxy that was resolved by forcefully closing all connections when a sleep/hibernate event was detected. This is only currently implemented for macOS, however.
You mention starting the proxy, so perhaps this is not the case, but it would be interesting to know.
Have you managed to find out any more about this issue in the meantime?
It isn't related to sleep or suspend, everything is the same even just after starting the proxy. During sleep the proxy doesn't work but as soon as the computer wakes from sleep the proxy continues to work.
My next tests are going to be:
- Connecting the computer running the proxy with ethernet cable instead of using wifi (the computer running email client already is connected with the cable).
- Try to install the same version of Window XP which is installed in the PC running email client in virtual machine (I am not sure if I'll find the correct ISO image - I have it somewhere but I installed these Windows back in 2007).
I once had a problem with DLNA server - after changing the ethernet cable everything worked well but the TV couldn't discover DLNA server despite browser and YouTube were working well. I think one of the ethernet cables was patch and the other one was crossover but I still don't know how that could affect only DLNA and nothing else (seems as that has something to do with UPnP - with one type of the cable UPnP didn't work).
I connected the computer running the proxy with ethernet cable instead of using wifi but nothing changed.
The blocking behaviour could be caused by unresolved connection failures – the proxy's connection to the server is initiated in blocking mode, which could cause it to hang. It does try to close all connections, including failed ones, though this might be less reliable on Windows.
Have you experienced this behaviour with the Python 3 version of the script, or just the Python 2 branch?
Yes, I read socket.recv() is a blocking function but still can't find out the reason what could cause the problem.
I am testing with the latest Python 3 version from the main branch all the time. I will continue with testing of Pyhon 2 branch as soon as Python 3 version will work without problems.
I'd like to capture the traffic in between laptop running email client and the router and in between the router and Windows 10 computer running the proxy but I couldn't find any ethernet hub to place it between PC running email client and the router and between the router end PC laptop running the proxy - seems as don't sell network hubs anymore.
I have two network switches but they don't have port mirroring (SPAN) and such swithes are quite expensive but there is LAN TAP device on Aliexpress which is not expensive and can be used as a passive Terminal Access Point. I am going to buy one of these to check where exactly the TCP packet containing attachment is lost. LAN TAP can be used for capturing the traffic in one direction only but that should be enough.
It might be worth trying the nonblocking SSL branch if you suspect the blocking socket could be anything to do with this. This branch takes advantage of recent improvements to local secure connections, and uses that approach for the remote server as well.
I've just tested the nonblocking SSL branch and everything is still the same - I can't send an email with attachment.
Now that you mentioned SSL - is there anything involving SSL that could affect sending the packets containing attachments?
I was searching for problems with sending attachment related to SSL and there is a support ticket for some software I've never heard of but here is what is written in the description:
PROBLEM
Track-It 2020.02: Cannot send email with attachment when using SSL/TLS protocol is selected in SMTP outgoing settings
When using SSL/TLS protocol for outgoing emails attachments are not sent successfully. Also report schedules fail to send the report as attachment.
According to the description of that problem it seems as there is something related to SSL that could under certain conditions prevent sending attachments.
This is an interesting find, and may conceivably even be the root cause here, but is ultimately not something that can be changed because SSL is mandatory for the connection to the remote server in all cases. You can disable local SSL for the proxy, but it is not enabled by default, so I'm guessing you're not using this configuration anyway.
You could perhaps test whether this is related to the proxy or not by testing connecting directly to the SMTP server from your client, as this can still be configured to accept basic authentication (see Google's documentation)?
You are right, I didn't enable local SSL.
Do you mean to test direct connection by creating app-specific password? That is a problem because that option could be enabled only after turning on 2-Step-Verification which is asking to give the phone number which I am not comfortable with. If once it would be impossible to use gmail without giving a phone number then I will stop using it.
BTW, I installed Python 3.4.3 in the PC where email client is running (that is the last version that could be used with Windows XP) and created a virtual environment (virtualenv didn't work because of failed to find interpreter for {discover} but venv did work well) to test running the proxy from the same computer as the email client but I couldn't install the requirements.
When I tried to upgrade PIP the process didn't finish (there was full screen of errors) and PIP stopped working so I had to delete the virtual environment and create another one. When I tried to install requirements without upgrading PIP there was an error Command "python setup.py egg_info" failed with error code 1 when building crypthography.
I just did a test to check if maybe some file from email client installation is missing or damaged. I created a new Windows XP virtual machine and transfered the whole folder containing the email client from the PC where email client is running to VM and in that case the email with the attachment was sent without problems.
But there is something I noticed. Eudora has progress bar which shows while sending emails and since the test email is very short and the attachment is a text file containing only one character the progress bar moves from 0 to 100% very quickly. However, the progress bar always stops for one second at exactly the same spot where in the case of sending from the PC it stops forever. And that spot is (probably) exactly the same spot when the TCP packet that is never delivered is to be transfered.
In other words - without stopping for a second the progress bar would move from 0 to 100% in about half a second or even faster. But it stops for about one second when it reaches about 90% and then continues - meaning there is something going on when the TCP packet containing the attachment has to be transfered.
And that is exactly how the progress bar always behaved.
I just did a test to check if maybe some file from email client installation is missing or damaged. I created a new Windows XP virtual machine and transfered the whole folder containing the email client from the PC where email client is running to VM and in that case the email with the attachment was sent without problems.
I think that underlying all of this there is the issue that Windows XP was released well before all of the current standard secure connection protocols were in use, and the workarounds to give it this capability are perhaps not totally reliable. I'm supportive of using the proxy to enable continued use of obsoleted devices/apps, but there are lower level things that it cannot help with.
Given that the issue here does not happen in a VM, and that it hasn't been possible to replicate it outside of the problematic setup so far, I think it might be time to put this to rest. I don't think there's any evidence that this issue is caused by the proxy at least.
I agree. You may close this issue. I will still continue to investigate the cause of the problem and report back when/if I find something useful.