child segfault

[ack]

we've hit a rather odd segfault while deploying throttle.

our config is pretty straightforward

import throttle;

sub vcl_miss {
  // this one'll be enforced before rps_hit
  if(req.url !~ "\.(jpg|jpeg|png|gif|ico|swf|css|js|html|htm)$") {
     if(throttle.is_allowed(req.http.host, "200req/s") > 0s) {
        error 429 "Calm down";
     }
  }
}

sub vcl_recv {
  ...

  if(req.url !~ "\.(jpg|jpeg|png|gif|ico|swf|css|js|html|htm)$") {
      if(throttle.is_allowed("ip:" + client.ip, "400req/s") > 0s) {
         error 429 "Calm down";
      }
  }

  if(req.url !~ "\.(jpg|jpeg|png|gif|ico|swf|css|js|html|htm)$") {
     if(throttle.is_allowed(req.http.host, "400req/s") > 0s) {
        error 429 "Calm down";
     }
  }

  ...
}

on inbound traffic, we see the following:

      Mar  5 03:18:51 pigpen-01 varnishd[23342]: Child (16092) said Child starts
      Mar  5 03:18:57 pigpen-01 kernel: [ 3130.146325] varnishd[16301]: segfault at 0 ip 00007fbe678f5080 sp 00007fbe02826298 error 4 in libvmod_throttle.so[7fbe678f4000+3000]
      Mar  5 03:18:57 pigpen-01 varnishd[23342]: Child (16092) died signal=11
      Mar  5 03:18:57 pigpen-01 varnishd[23342]: Child cleanup complete
      Mar  5 03:18:57 pigpen-01 varnishd[23342]: child (16508) Started
      Mar  5 03:18:57 pigpen-01 varnishd[23342]: Child (16508) said Child starts

what's particular vexing about this segfault is it appears to only happen on certain types of (virtualized) hardware (?)

we're running your plugin with these configs successfully w/o segfaults on aws, but the same code running in rackspace demonstrates the issue.

rackspace: Linux pigpen-01 3.2.0-24-virtual #37-Ubuntu SMP Wed Apr 25 10:17:19 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

aws: Linux headbox 3.2.0-23-virtual #36-Ubuntu SMP Tue Apr 10 22:29:03 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

any guidance on debugging this issue would be greatly appreciated.

strace of the child

      writev(12, [{"200 19      \n", 13}, {"PONG 1362458677 1.0", 19}, {"\n", 1}], 3) = 33
      poll([{fd=9, events=POLLIN}], 1, -1)    = 1 ([{fd=9, revents=POLLIN}])
      read(9, "ping\n", 8191)                 = 5
      writev(12, [{"200 19      \n", 13}, {"PONG 1362458680 1.0", 19}, {"\n", 1}], 3) = 33
      poll([{fd=9, events=POLLIN}], 1, -1)    = 1 ([{fd=9, revents=POLLIN}])
      read(9, "ping\n", 8191)                 = 5
      writev(12, [{"200 19      \n", 13}, {"PONG 1362458683 1.0", 19}, {"\n", 1}], 3) = 33
      poll([{fd=9, events=POLLIN}], 1, -1)    = 1 ([{fd=9, revents=POLLIN}])
      read(9, "ping\n", 8191)                 = 5
      writev(12, [{"200 19      \n", 13}, {"PONG 1362458686 1.0", 19}, {"\n", 1}], 3) = 33
      poll([{fd=9, events=POLLIN}], 1, -1)    = 1 ([{fd=9, revents=POLLIN}])
      read(9, "ping\n", 8191)                 = 5
      writev(12, [{"200 19      \n", 13}, {"PONG 1362458689 1.0", 19}, {"\n", 1}], 3) = 33
      poll([{fd=9, events=POLLIN}], 1, -1)    = 1 ([{fd=9, revents=POLLIN}])
      read(9, "ping\n", 8191)                 = 5
      writev(12, [{"200 19      \n", 13}, {"PONG 1362458692 1.0", 19}, {"\n", 1}], 3) = 33
      poll([{fd=9, events=POLLIN}], 1, -1 <unfinished ...>
      +++ killed by SIGSEGV +++

[nand2]

Hello!

I have at least a quick way to check and debug your problem. There is a known bug (https://github.com/nand2/libvmod-throttle/issues/3) that this vmod will segfault, if the given key is an empty string.

I see that you are doing:

if(throttle.is_allowed(req.http.host, "400req/s") > 0s) {

Could you try adding a prefix to our key, e.g.

if(throttle.is_allowed("host:" + req.http.host, "400req/s") > 0s) {

and check if it still segfaults? I believe you may get empty Host headers...

Nicolas