Stream parser throws away data if too much read from socket

[daggaz]

trafficstars

If the socket reads more data than requested (as in the SSL case), then stream parser can get more data than it requested.

This means that the some_bytes variable here can contain more than one frame's worth of data.

The frame parser then correctly reads one frame out of the data, but the rest of the data is thrown away.

[willmcgugan]

This is exactly what happens. We worked around it with by overriding the once method as follows:

    def once(self):
        """
        Performs the operation of reading from the underlying
        connection in order to feed the stream of bytes.
        We start with a small size of two bytes to be read
        from the connection so that we can quickly parse an
        incoming frame header. Then the stream indicates
        whatever size must be read from the connection since
        it knows the frame payload length.
        It returns `False` if an error occurred at the
        socket level or during the bytes processing. Otherwise,
        it returns `True`.
        """
        if self.terminated:
            return False

        try:
            b = self.sock.recv(self.reading_buffer_size)
        except (socket.error, OSError, pyOpenSSLError) as e:
            self.unhandled_error(e)
            return False
        else:
            if not self.process(b):
                return False

        return True

That fix appears to be good enough for the threaded client.

[hyperair]

I just spent the last few hours debugging this issue before realizing there was an open issue on it:

https://github.com/hyperair/WebSocket-for-Python/commit/bf5e1f07dc3c8a03c6ee927ecadbab2bf45b316d

I'll file a PR later.