leveldb icon indicating copy to clipboard operation
leveldb copied to clipboard

Bug report: Writing to wal succeeds, but memtable fails, which will lead to data inconsistency?

Open selfboot opened this issue 11 months ago • 1 comments

In a write operation, if WAL is written successfully but memtable is written unsuccessfully, will this lead to data inconsistency?

If a memtable write fails, LevelDB returns an error status to the caller, indicating that the write operation was unsuccessful.

However, the WAL still retains records of these operations. When the database is restored, these records will be reapplied to the MemTable, resulting in inconsistent data.

Related code

    // Add to log and apply to memtable.  We can release the lock
    // during this phase since &w is currently responsible for logging
    // and protects against concurrent loggers and concurrent writes
    // into mem_.
    {
      mutex_.Unlock();
      status = log_->AddRecord(WriteBatchInternal::Contents(write_batch));
      bool sync_error = false;
      if (status.ok() && options.sync) {
        status = logfile_->Sync();
        if (!status.ok()) {
          sync_error = true;
        }
      }
      if (status.ok()) {
        status = WriteBatchInternal::InsertInto(write_batch, mem_);
      }
      mutex_.Lock();
      if (sync_error) {
        // The state of the log file is indeterminate: the log record we
        // just added may or may not show up when the DB is re-opened.
        // So we force the DB into a mode where all future writes fail.
        RecordBackgroundError(status);
      }
    }

Or did I misunderstand? Can anyone help explain this, thank you very much.

selfboot avatar Jan 24 '25 03:01 selfboot

When I saw relative codes, I had the same questions.

Maybe, failures to write memtable are almost unlikely to occur. Only when memory is exhausted or codes added later have bugs, writting memtable fails. The process will oom if no memory, so you won't see the results returned by writting, although WAL has been written. Usually in line with expectations.

The method Add in MemTable return void, so people don't think there could be a mistake here.

13015517713 avatar May 18 '25 15:05 13015517713