abiosoft
Disk migration / upgrade disk size manually
Description
Hello,
is there some way of disk migration? The only thing I can find about disk ist the following:
NOTE: disk size cannot be changed after the VM is created.
While this is totally understandable, is there some way of migration? Like copy foo from the VM to the host via bar, delete VM, create new one and copy foo to VM at baz.
Since this is a Linux VM you have access to things like ssh and scp, one could copy files out using those, and then back in.
Since this is a Linux VM you have access to things like ssh and scp, one could copy files out using those, and then back in.
Do you have any idea about all the necessary files and folders? A very simple overview of the inner structure would help closing this issue.
It depends on what you want to save, if it's the docker volumes you can list them with a fdisk -l. Otherwise I wouldn't care too much, and just delete the VM to create a new one.
If it's primarily with regards to docker (or OCI) images, you can docker save to a .tar file and docker load after a reset.
If it's primarily with regards to docker (or OCI) images, you can
docker saveto a .tar file anddocker loadafter a reset.
Well, the main request for this that with Docker Desktop you can adjust the VM size without any data loss. Currently that's not possible with Colima so I asked for some sort of documentation for manual upgrading "without any data loss". But I assume that the images are the least required data, more important are the volumes for example.
hey @Morgy93 you should be able to use qemu-img for the task, e.g. qemu-img resize ~/.lima/colima/diffdisk +10G, the change needs vm restart
Oh my goodness, if that works could you add a PR for the docs @artygus ? We all thought you had to recreate. Of course only works for qemu.
Oh my goodness, if that works[...]
This requires lima 0.14+ (https://github.com/lima-vm/lima/pull/1126).
Also the original VM must have been created by at least lima 0.13 (or maybe also 0.14); otherwise it doesn't include the tools to grow the partition. You can check via ssh to see if your VM has the growpart and resize2fs utilities installed and on the PATH.
@jandubois thanks for the notes! one problem noticed is that lima reports the initial size in limactl list, are there plans to fix it?
one problem noticed is that lima reports the initial size in
limactl list, are there plans to fix it?
I'm not aware; please file an issue on the Lima repo!
I wonder what is the right size to report though? We should probably have options for both current size and maximum size, but determining those numbers could be slow-ish. But we can discuss this in a Lima issue.
I tried all of that today and created a walk through. This applies - at least - to macOS. Thanks to @tobi-pb for the help, insights and knowledge!
BACK UP ! – DO IT!
no further words required, I think 😉
TL;DR
If you blindly copy and paste and paths or device names are different, you're on your own! Better read the explanation … it's worth the minutes.
qemu-img resize ~/.lima/colima/diffdisk 150G
limactl shell colima
sudo apk add e2fsprogs-extra
sudo apk add cloud-utils-growpart
sudo growpart /dev/vdb 1
sudo resize2fs /dev/vdb1
vi ~/.lima/colima/lima.yaml
disk: 60GiB
--> disk: 150GiB
vi ~/.colima/default/colima.yaml
disk: 60
--> disk: 150
Now in detail, with a bit of explanation:
Changing QEMU Image first
qemu-img info ~/.lima/colima/diffdisk shows the current size of your colima disk image
image: /Users/username/.lima/colima/diffdisk
file format: qcow2
virtual size: 60 GiB (161061273600 bytes)
disk size: 58 GiB
cluster_size: 65536
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: false
refcount bits: 16
corrupt: false
extended l2: false
qemu-img resize ~/.lima/colima/diffdisk 150G will set your disk size to 150 GB, but that's not yet reflected within your container.
qemu-img info ~/.lima/colima/diffdisk verifies the updated virtual size (from 60 to 150)
image: /Users/username/.lima/colima/diffdisk
file format: qcow2
virtual size: 150 GiB (161061273600 bytes)
disk size: 58 GiB
cluster_size: 65536
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: false
refcount bits: 16
corrupt: false
extended l2: false
Lima work
First check the name of your lima virtual machine with
limactl list
NAME STATUS SSH VMTYPE ARCH CPUS MEMORY DISK DIR
colima Running 127.0.0.1:49503 qemu aarch64 4 16GiB 120GiB ~/.lima/colima
here the "NAME" is colima. We'll need that for the next step.
limactl shell colima will log you into your virtual machine
we have to install a few tools now, which are missing from alpine distribution:
sudo apk add e2fsprogs-extra
sudo apk add cloud-utils-growpart
sudo apk add parted
df -h the disk size still shows 60 GB …
…
/dev/disk/by-label/data-volume
58.8G 51.9G 3.9G 93% /usr/local
…
sudo parted will start parted. Now enter the command within parted:
print all
whill will output
/dev/vdb (161GB)
/dev/vdc (6947kB)
/dev/vda (176MB)
(parted) print all
Model: Virtio Block Device (virtblk)
Disk /dev/vdb: 161GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 1049kB 64.4GB 64.4GB primary ext4
There's a line starting with "Disk" in the middle. It shows you the disk's device name /dev/vdb/ and the table at the bottom with the number 1. Both is needed!
Leave parted with
quit
sudo growpart /dev/vdb 1 increase the partition size with device name and number, separated by a space!
CHANGED: partition=1 start=2048 old: size=125827072 end=125829120 new: size=314570719 end=314572767
sudo resize2fs /dev/vdb1 now also change the filesystem. The device name and number are now one single string! No space in between.
resize2fs 1.46.2 (28-Feb-2021)
Filesystem at /dev/vdb1 is mounted on /mnt/data; on-line resizing required
old_desc_blocks = 8, new_desc_blocks = 19
The filesystem on /dev/vdb1 is now 39321339 (4k) blocks long.
df -h verifies that our disk space did successfully increase.
…
/dev/disk/by-label/data-volume
147.4G 51.9G 88.9G 37% /usr/local
…
clean up
limactl list still shows the wrong disk size - that's just text, which we can change.
NAME STATUS SSH VMTYPE ARCH CPUS MEMORY DISK DIR
colima Running 127.0.0.1:57314 qemu aarch64 4 16GiB 60GiB ~/.lima/colima
edit ~/.lima/colima/lima.yaml and change the line
disk: 60GiB
to
disk: 150GiB
limactl list will now show the correct disk size.
NAME STATUS SSH VMTYPE ARCH CPUS MEMORY DISK DIR
colima Running 127.0.0.1:57314 qemu aarch64 4 16GiB 150GiB ~/.lima/colima
edit ~/.colima/default/colima.yaml and change the line
disk: 60
to
disk: 150
That makes sure that if you ever start over from scratch you'll start with the larger disk image right away.
I just did this on macOS ventura with limactl version 0.14.2 and colima version 0.5.2 and Docker version 23.0.1, build a5ee5b1dfc and qemu-img version 7.2.0
This should be possible now with the pre-release version i.e. brew install --head colima.
Simply increasing the disk size via the flag or config should resize the disk and reflect in the VM accordingly.
This should be possible now with the pre-release version i.e.
brew install --head lima.
This should have been possible since at least Lima 0.14.2. Beyond that it depends on the Alpine ISO including the right tools, so having created the VM with a recent version of colima should be more important.
This should have been possible since at least Lima 0.14.2.
@jandubois I'm referring to having the feature in-built. i.e. colima running the qemu-img resize automatically for the user when the disk size changes.
As for someone doing it manually, yeah it is possible since Lima 0.14.2.
The Alpine ISO already includes the right tools and grows the partition accordingly.
I'm referring to having the feature in-built. i.e. colima running the
qemu-img resizeautomatically for the user when the disk size changes.
But doesn't that mean you would need the head of colima and not lima? You wrote
brew install --head lima
Shouldn't that be
brew install --head colima