berserker
berserker copied to clipboard
Create possibility to generate large messages
trafficstars
In order to test the performance of stream processing frameworks, we need messages of 1MB and 10MB with a rate of 500, 5000 and 50000 messages per second.
With the current implementation, we can use:
- randomContentString() function to generate string of predefined size, with size of 1000000 characters berserker was able to create ~60 messages per sec
- to use a predefined string of large size, with a string of 1MB it took 15 minutes to start producing messages, berserker was able to produce ~300 messages per second
Two possible approaches:
- use the content of a predefined file as value for the field
- only once generate the string of desired length and reuse it