mod_maxminddb icon indicating copy to clipboard operation
mod_maxminddb copied to clipboard

Enhance documentation to discuss database updates

Open marcstern opened this issue 6 years ago • 5 comments

The documentation doesn't say if httpd must be stopped when updating the DB. That should be added: no restart, restart, stop/start?

marcstern avatar Nov 23 '18 12:11 marcstern

Yeah, I agree it would be useful to say something about this in the docs.

Reloading should be sufficient to see the new version.

horgh avatar Nov 24 '18 00:11 horgh

Parts of the DB are never reread from disk, even in case of swapping, etc. Otherwise, a running process could get some data about the new version while still having parts of the old one in memory.

marcstern avatar Nov 26 '18 08:11 marcstern

It shouldn't be the case that you have two conflicting versions of the database in use like you describe.

How are you updating the database? The files get memory mapped, so you'll want to be sure that when you update them you do not rewrite the files (e.g. with cp) but instead replace them (e.g. with mv on the same filesystem).

horgh avatar Nov 26 '18 16:11 horgh

Making a note which I hope is picked up by search bots.

If you have a process that updates all your servers from an internal mirror, something like:

/usr/bin/wget -q -r -nd -N -A '*.mmdb' -P /usr/share/GeoIP http://mirror.lan/share/geoip/

If there are changes, the mmdb file is rewritten. On a very active server, almost immediately your logs will be overwhelmed with error messages like these:

[mod_maxminddb]: Error getting data for '1.2.3.4': The MaxMind DB file's data section contains bad data (unknown data type or corrupt data)

Reloading the server will "fix" the problem, but the time between the update and the reload anything that relies on the environment variables is broken. As @horgh suggests, replacing (mv vs cp) should also work around the issue, but in a very active system there's still room for error.

I wonder if implementing the CheckCache, MemoryCache, and IndexCache options from mod_geoip2 would be worth the trouble.

h0tw1r3 avatar Apr 21 '21 04:04 h0tw1r3

Even with those options, the database would be corrupt while it is being copied over. You would need to atomically replace the database file, e.g., with mv on the same file system, to prevent the possibility of errors.

oschwald avatar Apr 21 '21 15:04 oschwald