fix: keep dirty memory bitmaps until snapshot is made

[RomanKovtyukh]

Changes

• Merge KVM bitmap of dirty pages into Firecracker internal one before going through dirty all pages and making a diff snapshot.
• Erase Firecracker bitmap only after all snapshots are done to avoid loosing dirty pages in case when error happens while we're making a snapshot.

Reason

KVM bitmap of dirty pages got erased at the moment we read it so if we read it and fail to use (for creating diff snapshot) we want to keep it somewhere until then next attempt of making snapshot.

Closes #4545.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check CONTRIBUTING.md.

PR Checklist

• [ ] If a specific issue led to this PR, this PR closes the issue.
• [ ] The description of changes is clear and encompassing.
• [ ] Any required documentation changes (code and docs) are included in this PR.
• [ ] API changes follow the Runbook for Firecracker API changes.
• [ ] User-facing changes are mentioned in CHANGELOG.md.
• [ ] All added/changed functionality is tested.
• [ ] New TODOs link to an issue.
• [ ] Commits meet contribution quality standards.

---

• [ ] This functionality cannot be added in rust-vmm.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6414a07) 81.53% compared to head (11a2a5f) 81.55%.

:exclamation: Current head 11a2a5f differs from pull request most recent head 2470ab4. Consider uploading reports for the commit 2470ab4 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4383      +/-   ##
==========================================
+ Coverage   81.53%   81.55%   +0.01%     
==========================================
  Files         241      241              
  Lines       29333    29364      +31     
==========================================
+ Hits        23916    23947      +31     
  Misses       5417     5417              

Flag	Coverage Δ
4.14-c7g.metal	76.94% <100.00%> (+0.02%)	:arrow_up:
4.14-m5d.metal	78.87% <100.00%> (+0.01%)	:arrow_up:
4.14-m6a.metal	77.99% <100.00%> (+0.02%)	:arrow_up:
4.14-m6g.metal	76.94% <100.00%> (+0.02%)	:arrow_up:
4.14-m6i.metal	78.86% <100.00%> (+0.02%)	:arrow_up:
5.10-c7g.metal	79.84% <100.00%> (+0.02%)	:arrow_up:
5.10-m5d.metal	81.52% <100.00%> (+0.01%)	:arrow_up:
5.10-m6a.metal	80.74% <100.00%> (+0.02%)	:arrow_up:
5.10-m6g.metal	79.84% <100.00%> (+0.02%)	:arrow_up:
5.10-m6i.metal	81.52% <100.00%> (+0.02%)	:arrow_up:
6.1-c7g.metal	79.84% <100.00%> (+0.02%)	:arrow_up:
6.1-m5d.metal	81.53% <100.00%> (+0.02%)	:arrow_up:
6.1-m6a.metal	80.74% <100.00%> (+0.02%)	:arrow_up:
6.1-m6g.metal	79.84% <100.00%> (+0.02%)	:arrow_up:
6.1-m6i.metal	81.52% <100.00%> (+0.02%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[roypat]

This got picked up in https://github.com/firecracker-microvm/firecracker/pull/4580