lima-vm
IPFS for downloading images, using decentralized peer-to-peer
Description
It should be possible to upload the images to IPFS, and then use such an address to download them p2p (by cid):
ipfs://bafybeig5sch22ecfox7gq724rz7uivydwvnnpuqdcnjz72iwelgtrakzui/ubuntu-24.04-server-cloudimg-amd64.img
CLI: https://docs.ipfs.tech/how-to/kubo-basic-cli/#install-kubo
GUI: https://docs.ipfs.tech/how-to/desktop-app/#install-ipfs-desktop
Implementation itself should be similar to ORAS: #2405
Setting up a hosting network is a completely different story.
Using an external ipfs tool on purpose, to cut down on dependencies.
It would be possible to use github.com/ipfs/kubo directly, as a "library".
Best case scenario, where both images and archives are found in local diskstore:
? Creating an instance "ipfs" Proceed with the current configuration
INFO[0000] Attempting to download the image arch=x86_64 digest="sha256:32a9d30d18803da72f5936cf2b7b9efcb4d0bb63c67933f17e3bdfd1751de3f3" location="https://cloud-images.ubuntu.com/releases/24.04/release-20240423/ubuntu-24.04-server-cloudimg-amd64.img"
Downloading the image (ubuntu-24.04-server-cloudimg-amd64.img)
453.00 MiB / 453.00 MiB [---------------------------------] 100.00% 340.48 MiB/s
INFO[0002] Used ipfs "bafybeig5sch22ecfox7gq724rz7uivydwvnnpuqdcnjz72iwelgtrakzui"
INFO[0002] Attempting to download the nerdctl archive arch=x86_64 digest="sha256:2c841e097fcfb5a1760bd354b3778cb695b44cd01f9f271c17507dc4a0b25606" location="https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-amd64.tar.gz"
Downloading the nerdctl archive (nerdctl-full-1.7.6-linux-amd64.tar.gz)
226.46 MiB / 226.46 MiB [---------------------------------] 100.00% 360.66 MiB/s
INFO[0004] Used ipfs "bafybeieipdaxd3fzy3j7syzzxdaqxramk65j7ajzcqgmi6b5jyq4jgbwue"
INFO[0004] Run `limactl start ipfs` to start the instance.
When ipfs (kubo) is not available, or image not found in IPFS, it would use HTTP...
In either case, the digest is verified before putting in the cache (with the url as key)
It is still possible to use ipfs:// as the primary, but now using a secondary cid:
images:
# Try to use release-yyyyMMdd image if available. Note that release-yyyyMMdd will be removed after several months.
- location: "https://cloud-images.ubuntu.com/releases/24.04/release-20240423/ubuntu-24.04-server-cloudimg-amd64.img"
arch: "x86_64"
digest: "sha256:32a9d30d18803da72f5936cf2b7b9efcb4d0bb63c67933f17e3bdfd1751de3f3"
cid: bafybeig5sch22ecfox7gq724rz7uivydwvnnpuqdcnjz72iwelgtrakzui
- location: "https://cloud-images.ubuntu.com/releases/24.04/release-20240423/ubuntu-24.04-server-cloudimg-arm64.img"
arch: "aarch64"
digest: "sha256:c841bac00925d3e6892d979798103a867931f255f28fefd9d5e07e3e22d0ef22"
cid: bafybeidyzm3zkljztw3xqqjjapcqzsr4odz4evnmk4kfhdmmgird6citbi
# Fallback to the latest release image.
# Hint: run `limactl prune` to invalidate the cache
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-amd64.img"
arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-arm64.img"
arch: "aarch64"
archives:
- location: "https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-amd64.tar.gz"
arch: "x86_64"
digest: "sha256:2c841e097fcfb5a1760bd354b3778cb695b44cd01f9f271c17507dc4a0b25606"
cid: bafybeieipdaxd3fzy3j7syzzxdaqxramk65j7ajzcqgmi6b5jyq4jgbwue
- location: "https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-arm64.tar.gz"
arch: "aarch64"
digest: "sha256:77c747f09853ee3d229d77e8de0dd3c85622537d82be57433dc1fca4493bab95"
cid: bafybeibptqoynryoxytxffgyuqjnmwpuujwp6wlps4spygapuwphzkeieq
# No arm-v7
# No riscv64
That means that it can share the digest (and the cache) with the https:// downloads.
The secret command to calculate the "cid" (IPFS's Content-ID) is:
ipfs add --cid-version=1 --only-hash --quieter
The digest (sha256) is still needed, as well*. CIDv1 is the new default.
* https://docs.ipfs.tech/concepts/content-addressing/#cids-are-not-file-hashes
If wanting to use only ipfs: for hosting, and not https: as the first URL:
Then probably --wrap-with-directory should be used, to keep the filename...
You can leave out the --quiet or --quieter option, if you need a progress bar.
But then the output is less suitable for automation, including an added prefix.
Can be implemented similar to the IPFS_GATEWAY in curl, rewrite the URL to a local bridge...
ipfs://bafybeig5sch22ecfox7gq724rz7uivydwvnnpuqdcnjz72iwelgtrakzui -> http://127.0.0.1:8080/ipfs/bafybeig5sch22ecfox7gq724rz7uivydwvnnpuqdcnjz72iwelgtrakzui
Currently all images are pulled from upstream, and none of them are providing IPFS CIDs.
Once lima starts to support image/archive mirroring (beyond cache), it could be revisited...