cole
SMTP.quit produces warning "Future exception was never retrieved"
I ported this code from my old implementation on standard smtplib module, and perhaps it is not correct for aiosmtplib. But i think this shouldn't matter.
import asyncio
import aiosmtplib
from email.mime.text import MIMEText
from email.header import Header
async def send_mail(host, login, password, from_addr, to_addr, subject, body_text):
msg=MIMEText(*body_text)
msg['Subject']=Header(subject, 'utf-8')
msg['From']=from_addr
msg['To']=to_addr
server=aiosmtplib.SMTP(host)
await server.connect()
await server.starttls()
await server.login(login, password)
try:
await server.sendmail(from_addr, [to_addr], msg.as_string())
except (aiosmtplib.SMTPRecipientsRefused, aiosmtplib.SMTPSenderRefused):
return 1
finally:
await server.quit()
#code for testing
async def start():
await send_mail('smtp.yandex.ru', '[email protected]', '1234', '[email protected]', '[email protected]', 'Testing', ['Hello', 'plain', 'utf-8'])
print('success')
#emulating work of other asyncio tasks...
await asyncio.sleep(3)
asyncio.run(start())```
```Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)] on win32
>>> import mail_test
success
>>> exit()
Future exception was never retrieved
future: <Future finished exception=SMTPServerDisconnected('Unexpected EOF received')>
aiosmtplib.errors.SMTPServerDisconnected: Unexpected EOF received```
If i comment two last lines of my function, it works good.
```>>> import mail_test
success
>>> exit()```
Last two lines meaning?
print('success')
#emulating work of other asyncio tasks...
await asyncio.sleep(3)
Last two lines meaning?
print('success') #emulating work of other asyncio tasks... await asyncio.sleep(3)
Because "Future exception was never retrieved" happens only if after calling quit passed at least a few seconds.
I get the same error with pytest test (using pytest-asyncio), for my tests it use gmail smtp with tls (if that matters)
setting PYTHONASYNCIODEBUG=1 this is the end of the output (everything before is not relevant)
...
File "/home/dev/development/newgateway/notifications_email_transport/src/notifications/transports/plugins/email/__init__.py", line 45, in send
response = await aiosmtplib.send(
File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/api.py", line 402, in send
result = await client.sendmail(sender, recipients, message)
File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/connection.py", line 151, in __aexit__
await self.quit()
File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/esmtp.py", line 236, in quit
response = await self.execute_command(b"QUIT", timeout=timeout)
File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/connection.py", line 420, in execute_command
response = await self.protocol.execute_command(*args, timeout=timeout)
File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/protocol.py", line 291, in execute_command
response = await self.read_response(timeout=timeout)
File "/home/dev/.virtualenvs/notifications_email_transport-ELxUtyWv/lib/python3.8/site-packages/aiosmtplib/protocol.py", line 268, in read_response
self._response_waiter = self._loop.create_future()
File "/usr/lib64/python3.8/asyncio/base_events.py", line 422, in create_future
return futures.Future(loop=self)
aiosmtplib.errors.SMTPServerDisconnected: Unexpected EOF received
By the way, the mail is sent, even with the exception.
OK, trying to have a look at the problem, I just discovered this does not happen every time, I just ran my test bench twice in a row, (full stop start, not twice the same test in code), first time was without problem, the second one has the error... so it something not so easy to reproduce to test...
I'm not 100% sure but, it seems that it depends when the garbage collectors kick because, if I put some print statement in SMTPProtocol.__del__ it print after the warning. So it means the last response waiter created while doing the QUIT command is left until the end of the run for a very fast run because the __del__ is not called before the end (garbage collection not done yet). It means also that, while the warning is boring and not super clean, it should be harmless for normal operations (when the __del__ is called before the very end of the software).
Deleting the references and manually calling gc.collect at the end of tests, I'm not able anymore to get the warning (but... it does not prove anything since it was not 100% reproducible)
I'm kind of stuck on a fix for this, since this isn't very reproducible (depends on when garbage collection happens relative to loop shutdown).
Note though that the warning should be harmless.
Hi,
I reproduced this issue:
Future exception was never retrieved
future: <Future finished exception=SMTPServerDisconnected('Connection lost')>
aiosmtplib.errors.SMTPServerDisconnected: Connection lost
I think the issue is not in my code and indeed is random and must be related with garbage collection.
edit: I use version 1.1.6
I can reliably reproduce this issue with the following code based on the docs.
Python 3.11.2 Version 3.0.0
import asyncio
from email.message import EmailMessage
import aiosmtplib
async def send_hello_world():
message = EmailMessage()
message["From"] = "[email protected]"
message["To"] = "[email protected]"
message["Subject"] = "Hello World!"
message.set_content("Sent via aiosmtplib")
await aiosmtplib.send(
message,
hostname="smtp.gmail.com",
port=587,
username="[email protected]",
password="used app password here"
)
print("Error happens after this print!")
asyncio.run(send_hello_world())
print("End of error!")
Oh nice, at least it's consistent on 3.11.2. Can you try the current main branch? I've made a change that fixes it for me.
I tried the new branch and the problem is still present with 3.11.2 on windows. Also tried with 3.10.12 on a linux VM and ran into the inconsistency problem, although it still happened.
However, I also found a way to avoid the problem:
import asyncio
from email.message import EmailMessage
import aiosmtplib
async def send_hello_world():
message = EmailMessage()
message["From"] = "[email protected]"
message["To"] = "[email protected]"
message["Subject"] = "Hello World!"
message.set_content("Sent via aiosmtplib")
await aiosmtplib.send(
message,
hostname="smtp.gmail.com",
port=587,
username="[email protected]",
password="used app password here"
)
print("Error doesn't happens after this print!")
loop = asyncio.get_event_loop()
loop.run_until_complete(send_hello_world())
print("No error!")
I'm still very new to concurrency and the asyncio lib, so I'm not really sure why this works...?
Reproduced on 3.10, 3.11 and 3.12 with aiosmtplib 3.0.0. In 3.12 some changes were made to the garbage collector, so it is strange that the problem appears in 3.11 and 3.10, perhaps changes were made to aiosmtplib itself.
I can confirm that I have that issue too. platform win32, python 3.11.6, aiosmtplib is 3.0.0
I'm still not sure how exactly that race condition is happening, but I did remove some __del__ logic in 3.0 on the protocol that seems to have caused it to happen. Please try current main branch if you're able to. If it works, I'll put out a patch release.
I'm still not sure how exactly that race condition is happening, but I did remove some
__del__logic in 3.0 on the protocol that seems to have caused it to happen. Please try currentmainbranch if you're able to. If it works, I'll put out a patch release.
can confirm, latest master version does not produce the error. win32, 3.11.6
I'm eagerly awaiting the patch. Thank you for your prompt responses and work :)
Unfortunately, sometimes the issue occurs again...
Future exception was never retrieved
future: <Future finished exception=SMTPServerDisconnected('Unexpected EOF received')>
python 3.11.6 aiosmtplib 3.0.1
No sorry, it's random...
I add that it's on Docker with image tiangolo/uvicorn-gunicorn:python3.11