Fix performance penalty when sending tars to daemon by optimizing layer size calculation
This PR addresses the performance penalty incurred by sending legacy formatted tars to the Docker daemon, as identified in the getLayerSize() function comments.
Problem
The current implementation has a significant performance bottleneck when creating tars for the Docker daemon:
- For each layer,
addLayerToTar()callslayer.Uncompressed()to get the layer data - It then calls
getLayerSize()to determine the uncompressed size for the tar header - When the layer size is unknown,
getLayerSize()callslayer.Uncompressed()again and reads through the entire layer withio.Copy(io.Discard, layerReader)just to calculate the size - This results in decompressing each layer twice and an expensive discard operation
Solution
Added an optimization that detects when all layers have known uncompressed sizes (typically after docker image save operations) and uses a more efficient code path:
-
canOptimizeLayerSizes(): Checks if all image layers have cached uncompressed sizes from previous docker save operations -
addLayerToTarOptimized(): Optimized version that uses cached sizes directly, avoiding the expensive size calculation -
Modified
addImageToTar(): Routes to the optimized path when beneficial
Performance Improvement
Before:
// For each layer:
layerReader := layer.Uncompressed() // First decompression
size := getLayerSize(layer) // Calls layer.Uncompressed() AGAIN
io.Copy(io.Discard, layerReader) // Expensive discard operation
After (when optimization applies):
// For each layer:
size := knownLayer.uncompressedSize // Use cached size
layerReader := layer.Uncompressed() // Single decompression
This eliminates redundant layer decompression and the expensive discard operation when working with images that have all layers downloaded from the daemon.
The optimization is automatically applied when beneficial and falls back to the original implementation when layer sizes are unknown, ensuring backward compatibility.
Fixes #276.
[!WARNING]
Firewall rules blocked me from connecting to one or more addresses
I tried to connect to the following addresses, but was blocked by firewall rules:
docker-images-prod.6aa30f8b08e16409b46e0173d6de2f56.r2.cloudflarestorage.com
- Triggering command:
/tmp/go-build2850353549/b355/remote.test -test.testlogfile=/tmp/go-build2850353549/b355/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.short=true(dns block)- Triggering command:
/home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js(dns block)eastus.data.mcr.microsoft.com
- Triggering command:
/tmp/go-build2850353549/b355/remote.test -test.testlogfile=/tmp/go-build2850353549/b355/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.short=true(dns block)myother-insecure-registry.com
- Triggering command:
/tmp/go-build2850353549/b355/remote.test -test.testlogfile=/tmp/go-build2850353549/b355/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.short=true(dns block)If you need me to access, download, or install something from one of these locations, you can either:
- Configure Actions setup steps to set up my environment, which run before the firewall is enabled
- Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.
@jabrown85 👋 This repository doesn't have Copilot instructions. With Copilot instructions, I can understand the repository better, work faster and produce higher quality PRs.
I can generate a .github/copilot-instructions.md file for you automatically. Click here to open a pre-filled issue and assign it to me. I'll write the instructions, and then tag you for review.