Random websocket wait() timeout with engineio

[Edvaytech]

In socket it give random timeout error

File "/usr/local/lib/python3.7/site-packages/eventlet/wsgi.py", line 566, in handle_one_response
result = self.application(self.environ, start_response)
File "/usr/local/lib/python3.7/site-packages/engineio/middleware.py", line 60, in call
return self.engineio_app.handle_request(environ, start_response)
File "/usr/local/lib/python3.7/site-packages/socketio/server.py", line 534, in handle_request
return self.eio.handle_request(environ, start_response)
File "/usr/local/lib/python3.7/site-packages/engineio/server.py", line 375, in handle_request
environ, start_response)
File "/usr/local/lib/python3.7/site-packages/engineio/socket.py", line 107, in handle_get_request
start_response)
File "/usr/local/lib/python3.7/site-packages/engineio/socket.py", line 147, in _upgrade_websocket
return ws(environ, start_response)
File "/usr/local/lib/python3.7/site-packages/engineio/async_drivers/eventlet.py", line 20, in call
return super(WebSocketWSGI, self).call(environ, start_response)
File "/usr/local/lib/python3.7/site-packages/eventlet/websocket.py", line 130, in call
self.handler(ws)
File "/usr/local/lib/python3.7/site-packages/engineio/socket.py", line 172, in _websocket_handler
pkt = ws.wait()
File "/usr/local/lib/python3.7/site-packages/eventlet/websocket.py", line 788, in wait
for i in self.iterator:
File "/usr/local/lib/python3.7/site-packages/eventlet/websocket.py", line 643, in _iter_frames
message = self._recv_frame(message=fragmented_message)
File "/usr/local/lib/python3.7/site-packages/eventlet/websocket.py", line 669, in _recv_frame
header = recv(2)
File "/usr/local/lib/python3.7/site-packages/eventlet/websocket.py", line 578, in _get_bytes
d = self.socket.recv(numbytes - len(data))
File "/usr/local/lib/python3.7/site-packages/eventlet/greenio/base.py", line 366, in recv
return self._recv_loop(self.fd.recv, b'', bufsize, flags)
File "/usr/local/lib/python3.7/site-packages/eventlet/greenio/base.py", line 360, in _recv_loop
self._read_trampoline()
File "/usr/local/lib/python3.7/site-packages/eventlet/greenio/base.py", line 331, in _read_trampoline
timeout_exc=socket_timeout('timed out'))
File "/usr/local/lib/python3.7/site-packages/eventlet/greenio/base.py", line 210, in _trampoline
mark_as_closed=self._mark_as_closed)
File "/usr/local/lib/python3.7/site-packages/eventlet/hubs/init.py", line 159, in trampoline
return hub.switch()
File "/usr/local/lib/python3.7/site-packages/eventlet/hubs/hub.py", line 298, in switch
return self.greenlet.switch()
socket.timeout: timed out

we are doing monkey patching as well

import eventlet
from socketio import WSGIApp
from edvay.settings import SOCKET_PORT
from edvay.sio_events import sio

eventlet.monkey_patch()
app = WSGIApp(sio)
eventlet.wsgi.server(eventlet.listen(('', int(SOCKET_PORT))), app)

from edvay.sio_app import sio


@sio.event
def connect(sid, environ):
    user_socket_id = environ['HTTP_USER_SOCKET_ID']
    print(sid, environ)
    sio.enter_room(sid, user_socket_id)


@sio.event
def disconnect(sid):
    # user_socket_id = environ['HTTP_USER_SOCKET_ID']
    # sio.leave_room(sid, user_socket_id)
    pass

from socketio import RedisManager, Server
from edvay.settings import REDIS_SOCKET_URL

mgr = RedisManager(REDIS_SOCKET_URL)  # , write_only=True)
sio = Server(client_manager=mgr, cors_allowed_origins='*')


def _send(user_socket_id, data, event):
    return sio.emit(event, {'data': data}, room=user_socket_id)


def sio_notification(user_socket_id, message):
    return _send(user_socket_id, message, 'notification')


def sio_chat(user_socket_id, data):
    return _send(user_socket_id, data, 'chat')

the version are

eventlet==0.25.1 redis==3.3.11 python-socketio==4.3.1

update: top traceback fixed width font format

[temoto]

Looking at this https://github.com/miguelgrinberg/python-engineio/blob/6daf0d19e572715543e426b9b4eda25e6aaf3f22/src/engineio/socket.py#L167 maybe this is expected behavior?

Please try to change server.ping_interval and server.ping_timeout

[Edvaytech]

thanks for your valuable reply @temoto i enable the logger=True, engineio_logger=True and i got this log

## ## ###, ### ### - - [16/Jun/2021 12:01:27] "OPTIONS /socket.io/?EIO=3&transport=polling&t=NeKihCV HTTP/1.1" 200 312 0.000306
socket_1         | f8349de5e781462bb1d9965d75ce4a53: Sending packet OPEN data {'sid': 'f8349de5e781462bb1d9965d75ce4a53', 'upgrades': ['websocket'], 'pingTimeout': 60000, 'pingInterval': 25000}
socket_1         | f8349de5e781462bb1d9965d75ce4a53 is entering room f74140f5-e856-4810-9866-939a5c0ce7eb [/]
socket_1         | f8349de5e781462bb1d9965d75ce4a53: Sending packet MESSAGE data 0
socket_1         | ## ## ###, ### ### - - [16/Jun/2021 12:01:28] "GET /socket.io/?EIO=3&transport=polling&t=NeKihCV HTTP/1.1" 200 398 0.001209

what values i should pass in ping_interval and ping_timeout.

update: log fixed width font format

[temoto]

@Edvaytech I'm not familiar with engineio, but judging by the name, I suppose you should set

• ping_interval = max normal time between messages
• ping_timeout = 10*RTT (for example 10s)

[Edvaytech]

@temoto can you please explain what is RTT so it will very helpful for me to set the value.

[temoto]

@Edvaytech typical time between request and response packet sans processing time. It's around distance / c plus network load and re-transmission delays.

https://en.wikipedia.org/wiki/Round-trip_delay

[Edvaytech]

@temoto

             Server(client_manager=mgr, cors_allowed_origins='*', logger=True, engineio_logger=True, ping_interval=25,
             ping_timeout=120)
             
           I gave ping timeout 120 and also try with 100 as well but still get timeout.

[temoto]

@Edvaytech are you also sending messages faster than every 120ms?

[temoto]

As much as I understand working of engineio:

• ping feature raises timeout because no other message was received so it thinks network doesn't work
• maybe you can set both ping_timeout and interval to 0 to disable this feature
• you should send other messages within ping_interval to help engineio understand that network works