rocksdb icon indicating copy to clipboard operation
rocksdb copied to clipboard

Avoid slow CI tests on MACOS_X using build flag

Open alanpaxton opened this issue 1 year ago • 0 comments

Build with MACOS_IGNORE_FULLFSYNC=1 to build a version of RocksDB which does not define HAVE_FULLFSYNC and is therefore much faster at some intensive open/close operations, such as are often found in testing.

Closes https://github.com/facebook/rocksdb/issues/13147 Closes https://github.com/facebook/rocksdb/issues/11035

by providing the option to build tests which can run much faster IF the tests are not involved with e.g. crash recovery

DO NOT use this in production, or in performance testing. HAVE_FULLSYNC was introduced in https://github.com/facebook/rocksdb/pull/9356 in order to prevent potential data loss / corruption issues

To invoke it, do this:

$ make clean jclean $ MACOS_IGNORE_FULLFSYNC=1 make -j12 rocksdbjava

This turns off the default -DHAVE_FULLFSYNC (on Mac only) on platform detection, which has a huge cost on open/close heavy workloads (e.g. unit testing) due to very conservative flush safety - using fcntl(fd_, F_FULLFSYNC).

The same option is also added to cmake build to turn off FULLFSYNC on MacOS build

alanpaxton avatar Jan 07 '25 09:01 alanpaxton