storm-php
storm-php copied to clipboard
Lazyshot
High CPU load
While I can use storm-php for my bolts I have very high CPU load. This is probably because of while(true) $line = trim($this->readLine()); ?
If this is the case should we implement usleep or something to ease off when no data is coming ?
The high CPU load is probably from the read loop. We should probably switch over to using stream_select and stream_get_line in the readLine method.
If you'd like to make the adjustments and see if this solves your CPU issue, then just submit a pull request. That would be greatly appreciated.
public function readLine()
{
$result = stream_select(array(STDIN), array(), array(), 2);
if ($result > 0)
{
return trim(stream_get_line(STDIN, 65535, "\n"));
}
else return "";
}
It may also be advantageous to do some research into using react
I'll test it tomorrow at work, @home it doest work since there's a bug in php 5.3.10 - stream_get_line(); And thanks for quick answer.
Do you know how tuples are sent to an php bolt ? Is it like tail -f file | php bolt.php ?
Since when I'm trying to debug the bolt behaviour with tail (and the DEBUG log generated form the bolt) I don't get the load with current code and the new one, but if I do: cat file | php bolt.php then the load is there.
So It might be that bolt got "disconnected" from the stream of tuples and therefor the load ?
That's essentially what it looks like. Just a flooded stream of json. Multilang Protocol for more information.
It sounds like we aren't checking for EOF anywhere in that loop. So we get an infinite loop at the end of the file. IIRC, cat throws an EOF, while tail -f does not. So you could see the differences there. Try adding an feof check in the loop somewhere to kill the php bolt.
Out of curiosity Is this still an issue? And to clarify, would it require react within storm-php or externally?
I'm doing research for a realtime analytics platform, and we will most likely make a symfony2 bundle encompassing this, as such could easily fork the code, but don't wan't to do the work double.
