Refactor dirty page tracking
[!NOTE] Replaces dirty page tracking with a thread-safe Tracker and updates UFFD, sandbox, and build components to use it, adding unregister support and minor API changes.
- Block / Tracking:
- Introduce
block.Tracker(thread-safe bitset) withHas,Add,Reset,Offsets,BitSet,Clone; unit tests added.- Extend
SlicerwithBlockSize().- UFFD:
- Change constructor to
uffd.New(memfile, socketPath); removeTrackedSliceDeviceusage.MemoryBackendAPI:Disable(ctx)now returns*block.Tracker.- Implement unregister via
UFFDIO_UNREGISTER;Uffd.Disableunregisters then returnsTrackerfromDirty().- Track faults via
block.Trackerwith bounded concurrency; expose offsets iterator in cross-process tests.- Buffer
Ready()channel.- Sandbox Integration:
- On pause, call
s.memory.Disable(ctx)and passdirty.BitSet()toMemoryDiffCreator.serveMemoryupdated for new UFFD constructor.- Build / Diff:
- Add
BlockSize()toNoDiff,localDiff, andStorageDiff.- Utils:
- Add
utils.TransformToiterator helper.Written by Cursor Bugbot for commit b6a0c3a4ba221432bc2ad32490194c45fd7c9fad. This will update automatically on new commits. Configure here.
I want to do a few more end to end tests, but it should be ready to merge right after https://github.com/e2b-dev/infra/tree/uffd-refactor
@codex review
💡 Codex Review
https://github.com/e2b-dev/infra/blob/116062b7729f46356f334abb7c158dea50876bfd/packages/orchestrator/internal/sandbox/uffd/userfaultfd/serve.go#L109-L113
Handle UFFD remove events after unregister The new
Disable(ctx)issuesUFFDIO_UNREGISTER, which causes the kernel to emitUFFD_EVENT_REMOVEnotifications on the userfaultfd. The serve loop still treats every non‑UFFD_EVENT_PAGEFAULTas an error and immediately returnsErrUnexpectedEventType. After disabling memory this will report a failure, tear down the goroutine and mark the backend as errored even though the event is expected. The loop should explicitly ignore or handleUFFD_EVENT_REMOVE(and possibly other control events) once unregistering is supported.ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Isn't UFFD_EVENT_REMOVE triggered by MADV_DONTNEED/MADV_REMOVE, which are not actually part of the UFFD unregister signalization—they are part of the UFFD memory management which we are not using. Not 100% sure though.
Summary
- Added the
UFFD_EVENT_REMOVEconstant, struct alias, and helper to surface remove event ranges for logging. packages/orchestrator/internal/sandbox/uffd/userfaultfd/constants.goL23-L112 - Updated the userfaultfd serve loop to ignore
UFFD_EVENT_REMOVEnotifications emitted after unregister, avoiding erroneous error handling. packages/orchestrator/internal/sandbox/uffd/userfaultfd/serve.goL109-L159
Testing
- ⚠️
go test ./packages/orchestrator/internal/sandbox/uffd/...(fails: go: command not found)
I'll do some last end-to-end testing and then we should be able to merge.