ngx_http_geoip2_module
ngx_http_geoip2_module copied to clipboard
leev
Nginx returns 502 'worker process exited on signal 7'
This is an odd one and I can't reproduce it reliably, but it happens often enough in our test/staging environments (but not production) that we've had to stop using the module. Signal 7 is SIGBUS and usually a module problem when it occurs in nginx.
What happens is that occasionally nginx will wedge on a request, returning a 502. The log contains this cryptic error:
worker process 8519 exited on signal 7
The only reason I know it's GeoIP2 is because when the nginx is recompiled without the geoip2 module, it goes away. I've tried both static and dynamic compilations and the result is the same.
I've put the nginx configure script and nginx.conf in a gist: https://gist.github.com/ysgard/bd8305af0689af5484c21230a2cbf0fc
The problem is odd because it only manifests on our environments which don't get a lot of load/traffic. On our production systems it runs without issue.
We had the same similar issue. Actually the problem was in process how mmdb (database) file was updated. We first download file in /tmp folder and then move it to /var (nginx use it from /var in our config). If /tmp and /var is placed on one file system then all works OK. Move is atomic in this case. But if /var and /tmp are placed on different file systems, move isn't atomic. Old mmdb file is already mmapped into nginx process virtual memory, and instead of inode change, move overwrites process memory leading to SIGBUS.
Interesting - we host the mmdb file on AWS's EFS filesystem, so this could be the issue. I'll try verifying.
I have the same issue. When i using geoipupdate to update our mmdb, nginx have some errors in error log, the nginx workers force recreate.
Verified that moving the mmdb files off of EFS and onto the root filesystem solved the issue for us.
