areg-sdk
areg-sdk copied to clipboard
Improve streaming data rate
Goal The example 20_pubdatarate (currently available in candidate branch) shows that the communication data rate is about 100-120 MB/s on the PC. This value should be increased to achieve at least 300 MB/s on the PC.
Motivation AREG SDK should handle high data rate without a problem (the minimum data rate on PC should be 300 MB/s). In most of the cases, the data rate should be dependent on CPU / HW, and less dependent on the code implementation. Currently nearly half of delay is because of the code. In the current implementation we serialize data in streaming object then copy data in the message. The creating event object for each target, copying data in the message, applying CRC cause delays.
Background information The events are instantiated for each client. Currently the data is serialized in the shared buffer, but then they are copied to each message. Analyze the possibility:
- check presence of the event listeners before starting serialization. Escape serialization if none is present.
- create an event only once for all targets. Instead, set the list of targets sorted by channels.
- the list should be handled by
mcrouter
, so that thesource
sends an event one time.
The current business logic serializes data in the shared buffer then copied in the message.
- analyze the possibility to unify messages (unify protocol) for IPC and local targets.
- make message generic and independent on target location / channel.
- Pay attention on mechanism of applying CRC. Maybe apply the CRC only on data and escape target / source.
Test changes using 20_pubdatarate