[WIP] Add disk2disk serialization foe ACE Algorithm
This PR adds a serialization routine that allows combination of dataset, graph, and mapping as per step (3) of #1404. The data will be combined on-the-fly while streamed from disk to disk while trying to minimize the required host memory.
It is build on top of #1404 . More details to follow.
CC @tfeher , @julianmi
This pull request requires additional validation before any workflows can run on NVIDIA's runners.
Pull request vetters can view their responsibilities here.
Contributors can view more details about this message here.
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.
Contributors can view more details about this message here.
@mfoerste4 has this already been rolled into the other ACE PR? Can we safely close this one?
@mfoerste4 has this already been rolled into the other ACE PR? Can we safely close this one?
Yes, this can be closed.