nydus
nydus copied to clipboard
feat(nydusify): add zero-disk streaming image transfer
Implement a new streaming copy mechanism for image transfer that enables direct source-to-target streaming without using intermediate disk storage.
Key features:
- Add
StreamCopyfor concurrent pull/push operations - Introduce
streamTransferManagerfor orchestrating streaming workflows - Support chunked transfer with progress tracking
- Add HTTP/HTTPS fallback with automatic retries
- Support concurrent layer processing with configurable concurrency
- Retain compatibility with traditional transfer mode
Performance improvements:
- Eliminate need for intermediate disk storage
- Lower memory footprint via streaming buffers
- Enable concurrent image layer processing
- Add progress logging for large transfers
Implementation details:
- Introduce
enableStreamTransferconfiguration option - Add
httpModeManagerfor connection switching logic - Optimize for both small and large file transfers
- Add robust error handling and retry logic
- Integrate with existing provider/resolver framework
This significantly improves performance for large image transfers by avoiding disk I/O bottlenecks.