Simulation failure - assertion failure in checkpoint

Open sfc-gh-jslocum opened this issue 2 years ago • 1 comments

bin/fdbserver -r simulation -f tests/rare/BlobGranuleRanges.toml -s 194789141 -b on --crash --trace_format json

on commit 60d62eca18fc6b8ef558901b80918957294c6977 from PR #9833

A subset of the stack trace returned by addr2line:

(inlined by) internal_error_impl(char const*, char const*, int) at /mnt/ephemeral/jslocum/buildcop/flow/Error.cpp:66
Optional<Standalone<StringRef>>::get() const & at /mnt/ephemeral/jslocum/buildcop/flow/include/flow/Optional.h:214
 (inlined by) (anonymous namespace)::ApplyMetadataMutationsImpl::checkSetCheckpointKeys(MutationRef) at /mnt/ephemeral/jslocum/buildcop/fdbserver/ApplyMetadataMutation.cpp:624
 (inlined by) (anonymous namespace)::ApplyMetadataMutationsImpl::apply() at /mnt/ephemeral/jslocum/buildcop/fdbserver/ApplyMetadataMutation.cpp:1321

Points to this being a race in ApplyMetadataMutations between a checkpoint mutation happening and the txn state store initializing its ss metadata on the line Tag tag = decodeServerTagValue(txnStateStore->readValue(serverTagKeyFor(ssID)).get().get());

Mar 29 '23 13:03 sfc-gh-jslocum

Can this be closed?

May 10 '23 17:05 jzhou77