Add fast path optimization for multi-threaded writes

[genezhang]

Fast Path Optimization for Multi-threaded Writes

(note: worked with Copilot based on performance benchmarking)

Summary

This PR adds a fast path optimization to LogMgr::setSN() that significantly improves multi-threaded write throughput by skipping the heavy writeMutex for common write operations.

Performance Improvement

Benchmarks show 50-135% improvement in write throughput:

Threads	Before (ops/sec)	After (ops/sec)	Improvement
1	~170,000	~260,000	+53%
2	~220,000	~340,000	+55%
4	~265,000	~430,000	+62%
8	~210,000	~445,000	+112%

Tested on Intel Core i5-1035G1 @ 1.00GHz, 8GB RAM, Linux

How It Works

The fast path bypasses the mutex when all these conditions are met:

1. Sequence number is NOT specified by user (will be auto-assigned)
2. Sequence number overwrite mode is disabled (allowOverwriteSeqNum = false)
3. Current log file exists, is not removed, and has write capacity

Thread Safety Guarantees

The optimization is safe because:

• Sequence number assignment uses atomic CAS in MemTable::assignSeqNum()
• MemTable operations use lock-free skiplist (skiplist_insert)
• Log file protection via LogFileInfoGuard refcounting prevents removal during write
• Graceful fallback to mutex-protected slow path if any condition fails

Code Flow

LogMgr::setSN(record)
    ├── Fast path checks
    │   ├── No user-specified seqnum?
    │   ├── Overwrite disabled?
    │   └── Log file valid and writable?
    │       └── YES → Direct write via lock-free MemTable
    │           └── Return immediately
    └── Slow path (mutex protected)
        └── Handle file rotation, overwrite, etc.

Changes

• src/log_mgr.cc: Added fast path optimization in setSN() (~35 lines)
• tests/jungle/mt_write_stress_test.cc: New multi-threaded write stress test
• tests/CMakeLists.txt: Added new test target

Testing

New Tests Added

• mt write stress (2/4/8/16 threads) - Concurrent writes with verification
• mt write read interleaved - Mixed read/write workload
• mt write with flush - Writes with background flusher
• mt write reopen - Durability test (write, close, reopen, verify)

Existing Tests

All existing tests pass:

• mt_test ✅
• sync_and_flush_test ✅
• basic_op_test (43/44 pass - 1 pre-existing failure)
• flush_stress_test ✅
• log_reclaim_stress_test ✅

Backward Compatibility

• No API changes
• No behavior change for edge cases (user-specified seqnum, overwrite mode)
• Falls back to original mutex-protected path when needed

Notes

This optimization is most beneficial for write-heavy workloads where:

• Multiple threads write concurrently
• Sequence numbers are auto-assigned (most common case)
• allowOverwriteSeqNum is disabled (default)

Workloads that specify sequence numbers or use overwrite mode will not benefit but will not be negatively affected.