chrome-devtools-protocol icon indicating copy to clipboard operation
chrome-devtools-protocol copied to clipboard

Clear Payload buffer after receiving

Open bch36 opened this issue 3 years ago • 0 comments

Wrench\Client keeps all received Payload objects in perpetuity. This results in a script eventually running out of memory if it's reading a large stream of data. Instead, after receiving any Payload objects buffered since the last call to receive(), clear the buffer of received Payloads. This is safe because they're never accessed again. Clear the buffer in ChromeDevtoolsProtocol\WebSocket\WebSocketClient to avoid upstream changes to Wrench\Client.

Additional info: Wrench\Client stores all received Payloads in Client::$received. In fact, if Payload::getPayload() is called (as it is in DevtoolsClient), memory usage will double because the received data is copied from the Payload's comprising Frames (Payload::$frames) into Payload::$buffer. The data is then stored a third time in the ReadResponse and returned to the original caller. The caller only has the ability to delete the third copy of the data in the ReadResponse. This results in memory usage growing quickly and perpetually as a long-lasting stream is read until the script eventually runs out of available memory.

For example, if a user generates a large PDF (printToPDF) using a transferMode of ReturnAsStream, they may call $session->io()->read() many times to fetch the generated PDF. As they do so, they'll likely write each block of data to a file. If the buffer of received Payloads is never cleared to free memory, the user is limited to receiving a PDF approximately 1/2 the size of the memory_limit. If the buffer of received Payloads is cleared regularly, then there is theoretically no limit to the size of PDF they can read.

bch36 avatar Mar 02 '21 18:03 bch36