Add support for snapshots

[JunBys]

I can't seem to find anything about a virtual machine container snapshot manager, which is readily available and available on vmware This is also very important for my work, which means that I need to create snapshots for different moments of the virtual machine to restore to a certain node location when needed! You should know what I'm talking about, I didn't explain otherwise

I've also looked through the historical issues and apparently haven't seen anyone making a similar request, would like to ask if this feature is of interest! I'm new to github so please excuse me if I'm doing something wrong, thanks

[ideologysec]

https://github.com/utmapp/UTM/pull/3893

[lnxbil]

I'm also missing this feature, yet I know how to do it manually, if you already have homebrew and qemu installed there:

• open the folder in which the virtual machine is (show in finder), right click the VM and show contents, change to Data subfolder. Open that in the Terminal.
• folder is usually something like this ~/Library/Containers/com.utmapp.UTM/Data/Documents/<VM Name>.utm/Data
• look for *.qcow2 files, that is the virtual disk (there may be more than one depending on your VM setup)
• runqemu-img to create and list the snapshot snapshot1

$ qemu-img snapshot F89035CE-DEF9-49D7-97F8-7EC0C2F3F9D9.qcow2 -c snapshot1

$ qemu-img snapshot F89035CE-DEF9-49D7-97F8-7EC0C2F3F9D9.qcow2 -l         
Snapshot list:
ID        TAG               VM SIZE                DATE     VM CLOCK     ICOUNT
1         snapshot1             0 B 2023-11-18 15:31:02 00:00:00.000          0

• do what you want to do with the VM
• revert back

$ qemu-img snapshot F89035CE-DEF9-49D7-97F8-7EC0C2F3F9D9.qcow2 -a snapshot1

[gitsnaf]

My virtual disks have the suffix img.

Creating a snapshot fails with:

WARNING: Image format was not specified for '95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.img' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
qemu-img: Could not create snapshot 'snapshot1': -45 (Operation not supported)

[lnxbil]

The snapshots I described only work with supported image types, e.g. qcow2. You may want to convert them to qcow2 to get all available features.

[gitsnaf]

Hm, after converting to qcow2 UTM doesn't find the image anymore. When I import the image file manually, then UTM tries to start the image but the window stays black. Die operating system inside the VM is not booting anymore.

[lnxbil]

How did you convert it? What does qemu-img info say about both images?

[gitsnaf]

$ qemu-img convert -f raw -O qcow2 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.img 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.qcow2

$ qemu-img info 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.img
image: 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.img
file format: raw
virtual size: 64 GiB (68719476736 bytes)
disk size: 17.9 GiB
Child node '/file':
    filename: 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.img
    protocol type: file
    file length: 64 GiB (68719476736 bytes)
    disk size: 17.9 GiB

$ qemu-img info 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.qcow2
image: 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.qcow2
file format: qcow2
virtual size: 64 GiB (68719476736 bytes)
disk size: 18.1 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false
Child node '/file':
    filename: 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.qcow2
    protocol type: file
    file length: 18.1 GiB (19473760256 bytes)
    disk size: 18.1 GiB

I also don't know how to create a new UTM VM with qcow2 as storage type.

[lnxbil]

Your virtual disk sizes are the same and everything looks like it should work.

I also don't know how to create a new UTM VM with qcow2 as storage type.

I don't know either, my default was qcow2, as I thought it would be and should be with QEMU.

This is my disk-part of the configuration, does it look the same or are there any other values?

	<key>Drive</key>
	<array>
		<dict>
			<key>Identifier</key>
			<string>F89035CE-DEF9-49D7-97F8-7EC0C2F3F9D9</string>
			<key>ImageName</key>
			<string>F89035CE-DEF9-49D7-97F8-7EC0C2F3F9D9.qcow2</string>
			<key>ImageType</key>
			<string>Disk</string>
			<key>Interface</key>
			<string>VirtIO</string>
			<key>InterfaceVersion</key>
			<integer>1</integer>
			<key>ReadOnly</key>
			<false/>
		</dict>
	</array>

[gitsnaf]

That is mine:

<key>Drive</key>
	<array>
		<dict>
			<key>Identifier</key>
			<string>363D013C-D19F-44B7-8079-82EA063BB118</string>
			<key>ImageName</key>
			<string>95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.qcow2</string>
			<key>ReadOnly</key>
			<false/>
		</dict>
	</array>

Because of the ID difference I created a new clone from a working vm and then without starting UTM I edited the config.plist und changed only the ending from .img to .qcow2 for the ImageName-Value. But it was not working. Then I added your lines, that means the keys ImageTyp, Disk and VirtIO. But again it did not work.

[lnxbil]

I just tried to setup a raw diskimage I just downloaded and it gets automatically converted by the GUI to qcow2 and boots up fine. You said you tried to import the image manually? Please copy the original img image outside, rename it to .raw and import the disk via the GUI (add new disk) and see if it also gets converted to qcow2.

Are you on the newest UTM app version?

[gitsnaf]

I did what you said but the file was not converted to to qcow2. UTM just copied the raw-File from the import location to the VM location.

[gitsnaf]

https://docs.getutm.app/settings-qemu/drive/drive/ says:

When importing a drive that is non-removable and not a raw image, the selected image will first be converted to the QCOW2 format.

So we cannot expect a conversion to qcow2 when importing a raw-image.

I'm using UTM 4.3.5. I will check for an update.

Update: I updated to UTM 4.4.4 but nothing changed. No qcow2-conversion when importing from a raw-file.

[lnxbil]

I'm also on 4.4.4 on a M2 MBP. I don't know what I should tell you, it does convert it to qcow2. I'm starting with up-to-date FreeBSD raw image:

$ qemu-img info FreeBSD-14.0-RELEASE-amd64.raw
image: FreeBSD-14.0-RELEASE-amd64.raw
file format: raw
virtual size: 6.03 GiB (6476638720 bytes)
disk size: 3.57 GiB
Child node '/file':
    filename: FreeBSD-14.0-RELEASE-amd64.raw
    protocol type: file
    file length: 6.03 GiB (6476638720 bytes)
    disk size: 3.57 GiB

created a new VM, delete its disk, import the raw and I ended up with a qcow2 file:

or as text:

$ tree -C
.
├── Data
│   ├── FreeBSD-14.0-RELEASE-amd64.qcow2
│   └── efi_vars.fd
└── config.plist

2 directories, 3 files


$ qemu-img info Data/FreeBSD-14.0-RELEASE-amd64.qcow2
image: Data/FreeBSD-14.0-RELEASE-amd64.qcow2
file format: qcow2
virtual size: 6.03 GiB (6476638720 bytes)
disk size: 3.27 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false
Child node '/file':
    filename: Data/FreeBSD-14.0-RELEASE-amd64.qcow2
    protocol type: file
    file length: 3.27 GiB (3506634752 bytes)
    disk size: 3.27 GiB

[gitsnaf]

Bingo - macOS seems not to work with qcow2 :(

├── Debian 11 (LDXE).utm
│   ├── Data
│   │   ├── 64C60E7D-25B1-4AF0-AFDF-B12F5688E431.qcow2
│   │   └── efi_vars.fd
│   └── config.plist
├── Public
│   └── README.txt
├── Windows 11 21H2.utm
│   ├── Data
│   │   ├── 2FDC55C0-1D37-4551-9307-E5D451A60848.qcow2
│   │   └── efi_vars.fd
│   └── config.plist
├── macOS Monterey 12.6.1.utm
│   ├── Data
│   │   ├── 66943B94-B4F3-48EB-9125-9C5546353B2C.img
│   │   └── AuxiliaryStorage
│   └── config.plist
└── macOS Ventura 13.6.1.utm
    ├── Data
    │   ├── 95E3BDD3-62C7-4CA3-8FE1-BC046EBFC26A.img
    │   └── AuxiliaryStorage
    ├── config.plist
    └── screenshot.png

[ideologysec]

Apple Virtualization doesn't support qcow2 images for macOS

[lnxbil]

Apple Virtualization doesn't support qcow2 images for macOS

That sounds reasonable, qcow2 is the default format for QEMU. So, if you need snapshots, maybe take on inside of the VM with APFS?

[gitsnaf]

Snapshots with APFS are not sufficient. I want to restore the whole system. So I think I will have to clone the VM every time I need a snapshot. Not perfect, but I can live with it.

[eqoram]

This is the one thing that is holding me back of using UTM.

[esaruoho]

i'm here because ChatGPT v4 hallucinates that UTM already has a Snapshot creation, restoring and deletion method. It's wild the kind of stuff that these LLMs will hallucinate. I was fully expecting this feature to be here, but instead I gotta keep a master VM image and clone it and run. and then delete and make another clone. hopefully this type of feature will land eventually :+1:

[lnxbil]

i'm here because ChatGPT v4 hallucinates that UTM already has a Snapshot creation, restoring and deletion method. It's wild the kind of stuff that these LLMs will hallucinate. I was fully expecting this feature to be here, but instead I gotta keep a master VM image and clone it and run. and then delete and make another clone. hopefully this type of feature will land eventually 👍

Without knowing what exactly ChatGPTv4 said/hallucinated, it may be technically correct: you can create snapshots of some virtual disk as shown in this comment.

[esaruoho]

Without knowing what exactly ChatGPTv4 said/hallucinated, it may be technically correct: you can create snapshots of some virtual disk as shown in this comment.

hallucination1 (excerpt)

Using Snapshots:
Once macOS Sonoma is installed and you have a clean state, you can use
UTM’s snapshot feature to save this state.
After installing and using an app, you can revert to the snapshot to return
to the clean state without the app installed.

then i asked it a followup question on how exactly to do that and it said

Q: thanks, utm worked like a charm. is there some way for you to explain the snapshot feature

ChatGPT
I'm glad to hear that UTM worked well for you! The snapshot feature in UTM
is a powerful tool that allows you to save the current state of your virtual
machine (VM) at any given point in time. This feature is incredibly useful for
testing software, updates, or configurations, as it enables you to quickly
revert the VM back to a previous state without needing to reinstall the OS
or the applications.

How Snapshots Work
When you take a snapshot, UTM captures the entire state of the VM,
including the contents of its memory, the settings, and the current
state of all the disks attached to the VM. This snapshot can be stored
and later restored, allowing the VM to return exactly to the state it was
in when the snapshot was taken.

Using Snapshots in UTM
To use the snapshot feature in UTM, you can follow these general steps:

Open Your VM in UTM: Start UTM and select the VM you want to work
with from the list of available virtual machines.

Take a Snapshot:
- Ensure your VM is running or paused; you cannot take a snapshot of
a VM that is completely powered off.
- Access the VM's control panel or menu options where you manage
the VM settings and controls.
- Look for an option labeled "Snapshots", "Manage Snapshots", or something
similar. The exact wording might vary depending on the version of
UTM you are using.
- Click to take a new snapshot. You may be prompted to give the snapshot
a name or description, which can be helpful for keeping track of multiple snapshots.

Restoring a Snapshot:
Navigate back to the snapshots section in the UTM interface.
You'll see a list of all snapshots you've taken. Select the
snapshot you want to restore.
Confirm the restoration. The VM will revert to the exact
state it was in when that snapshot was taken. Any changes
made to the VM after the snapshot was taken will be lost.

Things to Keep in Mind
Storage Space: Snapshots can consume a significant amount of
disk space, especially if you take many snapshots or if the VM
has a lot of data. It's a good practice to delete snapshots you no
longer need.
Performance: While a snapshot is active (i.e., you have reverted
to a specific snapshot and are running the VM from there),
performance might be slightly affected depending on your
system's specifications and the amount of changes since the
snapshot was taken.
Snapshot Management: Regularly review your snapshots,
deleting old or unnecessary ones to free up space and keep
your snapshot list manageable.

Snapshots are an essential feature for safely testing software,
updates, or configurations, offering a straightforward way to
revert to a known good state without the hassle of manual
backups or reinstalls.

[lnxbil]

hallucination1 (excerpt)

Thank you for reporting back. Yes, this is completely hallucinated, especially the part:

you cannot take a snapshot of a VM that is completely powered off

That is the only way to create consistent snapshots.

[callumgare]

I'm not seeing any reference to this project in this issue so linking to it here: https://github.com/Metamogul/UTM-Snapshot-Manager

Seems like @Metamogul has put together a nice little PoC GUI for how UTM could manage snapshots. I'd be great to have something like this integrated into UTM directly :)

[cgxxv]

Hi utmapp team. Do we have the snapshot feature right now? I really need it. It's so important for a virtual machine. Even through we have the qemu-img command, but it is not as good as a GUI operation.

[athalas]

I'm not seeing any reference to this project in this issue so linking to it here: https://github.com/Metamogul/UTM-Snapshot-Manager

Seems like @Metamogul has put together a nice little PoC GUI for how UTM could manage snapshots. I'd be great to have something like this integrated into UTM directly :)

I think that's a brilliant suggestion, +1 to that!