python-websockets
Possible bug when running sync in free-threading python
Hi,
I'm using websockets with free-threading python (3.14rc3) - ie with GIL disabled - and I occasionally have this runtime error when closing a connection:
File "/home/isma/bigrob/bigrob/api/poly.py", line 801, in thread_close_ws_session
session.ws_session.close() ~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/home/isma/bigrob/.venv/lib/python3.14t/site-packages/websockets/sync/connection.py", line 588, in close with self.send_context(): ~~~~~~~~~~~~~~~~~^^
File "/home/isma/.local/share/uv/python/cpython-3.14.0rc3+freethreaded-linux-x86_64-gnu/lib/python3.14t/contextlib.py", line 148, in __exit__
next(self.gen)
~~~~^^^^^^^^^^
File "/home/isma/bigrob/.venv/lib/python3.14t/site-packages/websockets/sync/connection.py", line 1009, in send_context
self.close_socket()
~~~~~~~~~~~~~~~~~^^
File "/home/isma/bigrob/.venv/lib/python3.14t/site-packages/websockets/sync/connection.py", line 1072, in close_socket
self.acknowledge_pending_pings()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/isma/bigrob/.venv/lib/python3.14t/site-packages/websockets/sync/connection.py", line 741, in acknowledge_pending_pings
for pong_waiter, _ping_timestamp, ack_on_close in self.pong_waiters.values():
~~~~~~~~~~~~~~~~~~~~~~~~^^
RuntimeError: dictionary changed size during iteration
Hasn't been easy to replicate it - happens rarely but just flagging it in case it is something you know the answer already
Thank you for reporting this problem. The error suggests a race condition between connection termination (when this code runs) and sending a ping or receiving a pong. I think there could be a bug, regardless of whether the free-threading version is used or not, and free-threading merely makes it more likely.
That's what I think might be causing it (I'm using version 15.0.1 with keepalive btw)
When the keepalive thread calls self.ping(), it modifies the pong_waiters dict here on line 656:
pong_waiter = threading.Event()
self.pong_waiters[data] = (pong_waiter, time.monotonic(), ack_on_close)
self.protocol.send_ping(data)
If that happens at the same time I'm closing the connection, there is a chance that this iterator below will raise the exception as the length of the dictionary changed during the iteration:
(acknowledge_pending_pings: line 742)
for pong_waiter, _ping_timestamp, ack_on_close in self.pong_waiters.values():
if ack_on_close:
pong_waiter.set()
This is very unlikely to happen with the GIL enabled becaused only one code block runs at a time and python usuaslly doesn't switch threads in the middle of a block like that (but it is not impossible). WIthout the GIL it is quite possible
I guess the "easy" fix could be to use the protocol_mutex lock you have elsewhere in the code but not sure how that could interact with other places that have the lock. Or maybe you could use an event to tell the keepalive thread that it shouldn't send new pings when we are trying to close a connection
Another workaround could be to have acknowledge_pending_pings iterate in a copy of self.pong_waiters - this way we know this exception won't be raised but then any ping that triggers while we are closing the connection would be lost
Yes, I agree, those are the main options to fix the problem, and a last option is to add a lock specifically for serializing operations on pong_waiters (because reusing protocol_mutex will probably create too much risk of deadlock).