pion
Performance measurement tool
Summary
It would be great to have a tool to measure general performance. Developing something similar to iperf would enable evaluating the performance in countless scenarios (loopback over memory only, loopback over IP, specific delay, packet loss, packet duplication, packet reordering, ...).
Motivation
Compare performance against other stacks and determine spots for optimisation.
Notes
It may be a good idea to split the tool up for usage over raw sockets and for loopback usage over memory only. The former allows for interop and network performance testing while the latter can be used to test CPU usage.
The test protocol should be as simple as possible and specifics are up for discussion.
Hey @lgrahl it is great to see you here :)
That would be awesome. It would be great to suss out bugs and could help help fuzz all the implementations against each other. That would really kill the monoculture argument quick.
If we have 3 versions running against each other and fixing/communicating stuff.
We really need to start the conversation with Tim Panton (and others) and get the Community RTC project going. First step is a name though :)
I'd really like to see numbers for SCTP's throughput/performance. I am running into issues myself where I get inexplicably low throughput using webrtc datachannels and being able to point to a benchmark where I can run myself, under ideal conditions, what the throughput can be - it would allow me (and other users) to understand where the bottlenecks are in their own code (would also flush out potential issues in pion/sctp itself)
I just filed #181 and am looking for ways to recreate it. I'm thinking of using docker-compose to start two dockers: one to run a simple pion-based echo server with and the other with a browser test environment based on headless chrome & karma. Here's how I see the testing suite:
- set the network environment
- start the clock
- generate random traffic
- ensure it gets all the traffic back
- stop the clock
