getNodeFat Missing Node State Tree error

[vlntb]

High Level Overview of Change

• In-memory cache is split between multiple instances of TaggedCache (i.e. LedgerHistory, TreeNodeCache, AcceptedLedgerCache and a few more).
• There is no synchronisation between the parts of the in-memory cache during the sweep.
• The cached item expiry time depends on the cache size and its relation to the cache target size. As a result, instances of TaggedCache can have different expiry thresholds.
• The Missing Node: State Tree error manifests itself in the following sequence of events:

1. processLedgerRequest triggers getLedger, which checks LedgerHistory
2. The logic that follows then assumes that if the ledger is present in the LedgerHistory, then ledger contents is also preserved in the other cache - TreeNodeCache.
3. However, since the expiry for the TreeNodeCache can be calculated differently, it could have been already removed from memory by the sweep function.

p.s. Ordering in the Application::doSweep is done correctly in this case (first TreeNodeCache, second - LedgerHistory), but this does not help because of the difference in the sweep expiry time.

Context of Change

Change the Log level to warning and monitor its frequency. If we start seeing frequent occurrences of this event, this might indicate that the target age should be increased.

Type of Change

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [x] Refactor (non-breaking change that only restructures code)
• [ ] Performance (increase or change in throughput and/or latency)
• [ ] Tests (you added tests for code that already exists, or your new feature included in this PR)
• [ ] Documentation update
• [ ] Chore (no impact to binary, e.g. .gitignore, formatting, dropping support for older tooling)
• [ ] Release

API Impact

• [ ] Public API: New feature (new methods and/or new fields)
• [ ] Public API: Breaking change (in general, breaking changes should only impact the next api_version)
• [ ] libxrpl change (any change that may affect libxrpl or dependents of libxrpl)
• [ ] Peer protocol change (must be backward compatible or bump the peer protocol version)

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 79.1%. Comparing base (8f2f531) to head (ab8fc2d). Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/xrpld/overlay/detail/PeerImp.cpp	0.0%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff            @@
##           develop   #5455     +/-   ##
=========================================
- Coverage     79.1%   79.1%   -0.0%     
=========================================
  Files          817     817             
  Lines        71702   71703      +1     
  Branches      8257    8259      +2     
=========================================
- Hits         56694   56690      -4     
- Misses       15008   15013      +5     

Files with missing lines	Coverage Δ
src/xrpld/overlay/detail/PeerImp.cpp	3.7% <0.0%> (ø)

... and 3 files with indirect coverage changes

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.