Glean icon indicating copy to clipboard operation
Glean copied to clipboard

[lmdb #7] Optionally mount restored DBs using squashfs

Open simonmar opened this issue 1 month ago • 1 comments

The new config flag db_lmdb_restore_unpack (default true) controls unpacking of the DB when it is restored. If this flag is set to false, then the DB will not be unpacked by restore, instead we just keep the squashfs file and attempt to mount it when the DB is opened. This allows storing compressed DBs and decompressing on the fly. The DB is unmounted when closed, so we're only keeping mounts for the subset of DBs that are being actively used. Mounting/unmounting are very fast, so this shouldn't impact open/close latency.

The overhead for Glass to access a compressed DB this way seems to be about 10% the first time, and 0% after the uncompressed pages are cached. This was using the current default compression algorithm of zstd level 8.

Currently using squashfuse_ll to mount which is supposed to be a little faster than squashfuse and not quite as fast as a native mount -t squashfs, but the latter requires the server to be running as root.

simonmar avatar Jan 13 '26 16:01 simonmar