Performance when using rocky as a LB?

[Unitech]

Hello h2non,

I'm trying to use rocky as a loadbalancer, before digging into using http middlewares for traffic replay.

I did a benchmark before going further with this code:

var rocky = require('rocky')

var proxy = rocky({ forwardHost:true })

proxy
  .balance(['http://localhost:8001']);

proxy.routeAll();

proxy.listen(3000);

Then the test http application listening on port 8001:

var http = require('http');

var server = http.createServer(function(req, res) {
  res.writeHead(200);
  res.end('hey');
}).listen(process.env.PORT || 8001, function() {
  console.log('App listening on port 8001');
});

I did a benchmark with wrk. The first one without rocky and the second by hitting rocky for traffic balancing:

unitech@t450: ~/rocky-lb/wrk
>>> wrk -c 100 -t 100 -d 10 http://localhost:8001
Running 10s test @ http://localhost:8001
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.11ms    1.06ms  30.05ms   96.07%
    Req/Sec   246.53     25.70   494.00     96.38%
  246977 requests in 10.10s, 28.50MB read
Requests/sec:  24453.03
Transfer/sec:      2.82MB

unitech@t450: ~/rocky-lb/wrk
>>> wrk -c 100 -t 100 -d 10 http://localhost:3000
Running 10s test @ http://localhost:3000
  100 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    66.87ms   13.40ms 179.84ms   77.24%
    Req/Sec    15.03      5.13    30.00     98.57%
  14997 requests in 10.10s, 1.87MB read
Requests/sec:   1484.76
Transfer/sec:    189.95KB

The result is quite different with and without rocky. Without rocky (builting node http server) I get 246000 requests processed in 10 seconds. By using rocky It goes to 14997 requests in 10 seconds, 16x less performant than the raw http server.

Do you have any tips on enhancing overall performance? Aside from performance, rocky is clearly robust!

Thanks

[Unitech]

Just did a CPU profiling while benchmarking

[Unitech]

I did some research on the toppic of request proxying/loadbalancing

https://gist.github.com/Unitech/e27cca4e5b906be25c7dc4356126ef97

Doing a simple tcp proxying is the more efficient data forwarding:

var net = require('net');

var targets = [{
  host: '127.0.0.1',
  port: 8001
}, {
  host: '127.0.0.1',
  port: 8002
}];

var rri = 0;

var server = net.createServer(function(socket) {
  var target_process = targets[rri++ % targets.length];
  console.log('Redirecting connection to %s:%s',
              target_process.host,
              target_process.port);
  var proxy = net.createConnection(target_process);

  socket.setNoDelay(true);
  proxy.setNoDelay(true);

  proxy.pipe(socket).pipe(proxy);

  socket.on('error', function(err) {
    console.error(err);
    socket.end();
  });

  proxy.on('error', function(err) {
    console.error(err);
    socket.end();
  });
});

server.listen(7000);

[Unitech]

I guess this does not support websocket upgrade

[h2non]

Thanks for the comments. I will answer you asap with some metrics too.