virtnbdbackup icon indicating copy to clipboard operation
virtnbdbackup copied to clipboard
verified publisher icon abbbi

Unable to detect disks suitable for backup when backing up VM with network based disks

Open steeldomejeff opened this issue 4 months ago • 20 comments

Version used virtnbdbackup 2.38 ceph 19.2.3

Describe the bug Unable to detect disks suitable for backup when backing up VM with RBD (ceph) based disks

Expected behavior Backup should proceed

Hypervisor and/or virtual machine information:

  • OS: Rocky 9.6
  • HV type Libvirt 10.10/KVM
  • virsh dumpxml for VM:

Logfiles: virtnbdbackup --start-domain -U 'qemu+ssh://root@testsystem1-1/system' --nbd-ip 'testsystem1-1' --nbd-port '29714' --ssh-user 'root' --checkpointdir '/virtnbdbackup/machines/backups/6b9f10d7-09a7-4275-8766-f69605bb08ce/_checkpoints' -d 'baseos-mtwcgtvs' -l 'auto' -o '/virtnbdbackup/machines/backups/6b9f10d7-09a7-4275-8766-f69605bb08ce/2025-10/11a5894d-5d4f-4aa8-9f74-81573882c0ea' -e --raw

[2025-10-21 19:19:07] INFO lib common - printVersion [main]: Version: 2.38 Arguments: /usr/bin/virtnbdbackup --start-domain -U qemu+ssh://root@testsystem1-1/system --nbd-ip testsystem1-1 --nbd-port 29714 --ssh-user root --checkpointdir /virtnbdbackup/machines/backups/6b9f10d7-09a7-4275-8766-f69605bb08ce/_checkpoints -d baseos-mtwcgtvs -l auto -o /virtnbdbackup/machines/backups/6b9f10d7-09a7-4275-8766-f69605bb08ce/2025-10/11a5894d-5d4f-4aa8-9f74-81573882c0ea -e --raw [2025-10-21 19:19:07] INFO root virtnbdbackup - main [main]: Backup level: [auto] [2025-10-21 19:19:07] INFO root virtnbdbackup - main [main]: Store checkpoints in: [/virtnbdbackup/machines/backups/6b9f10d7-09a7-4275-8766-f69605bb08ce/_checkpoints] [2025-10-21 19:19:07] INFO virt client - _connect [main]: Connected to remote host: [testsystem1-1] [2025-10-21 19:19:07] INFO root virtnbdbackup - main [main]: Libvirt library version: [10010000] [2025-10-21 19:19:07] INFO root virtnbdbackup - main [main]: NBD library version: [1.20.3] [2025-10-21 19:19:07] INFO root check - targetDir [main]: Backup mode auto, target folder is empty: executing full backup. [2025-10-21 19:19:07] ERROR virt client - getDomainDisks [main]: Unable to detect disk volume type for disk [vda] [2025-10-21 19:19:07] WARNING root virtnbdbackup - main [main]: Emulated TPM device attached: User action required. [2025-10-21 19:19:07] WARNING root virtnbdbackup - main [main]: Please manually backup contents of: [/var/lib/libvirt/swtpm/11a5894d-5d4f-4aa8-9f74-81573882c0ea/] [2025-10-21 19:19:07] INFO root virtnbdbackup - main [main]: Only raw disks attached, switching to backup mode copy. [2025-10-21 19:19:07] ERROR root virtnbdbackup - main [main]: Unable to detect disks suitable for backup. [2025-10-21 19:19:07] INFO root metadata - backupConfig [main]: Saving VM config to: [/virtnbdbackup/machines/backups/6b9f10d7-09a7-4275-8766-f69605bb08ce/2025-10/11a5894d-5d4f-4aa8-9f74-81573882c0ea/vmconfig.copy.xml] [2025-10-21 19:19:07] INFO root metadata - backupBootConfig [main]: Save additional boot config [loader] to: [/virtnbdbackup/machines/backups/6b9f10d7-09a7-4275-8766-f69605bb08ce/2025-10/11a5894d-5d4f-4aa8-9f74-81573882c0ea/OVMF_CODE.secboot.fd] [2025-10-21 19:19:07] INFO root metadata - backupBootConfig [main]: Save additional boot config [nvram] to: [/virtnbdbackup/machines/backups/6b9f10d7-09a7-4275-8766-f69605bb08ce/2025-10/11a5894d-5d4f-4aa8-9f74-81573882c0ea/baseos-mtwcgtvs_VARS.fd]

Workaround: None found.

steeldomejeff avatar Oct 22 '25 20:10 steeldomejeff

w/o vm configuration file im not able to tell what the issue here personally i dont use ceph/rbd direct attached devices for my virtual machines.

tool is unable to parse the disks from the xml config, so this vm seems to have a disk configuration type ive never experienced before:

getDomainDisks [main]: Unable to detect disk volume type for disk [vda]

abbbi avatar Oct 22 '25 20:10 abbbi

It looks like it dropped the XML data I included in the ticket...

Here is the section of the xml dump:

On Wed, Oct 22, 2025 at 4:49 PM Michael Ablassmeier < @.***> wrote:

abbbi left a comment (abbbi/virtnbdbackup#286) https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3434158630

w/o vm configuration file im not able to tell what the issue here personally i dont use ceph/rbd direct attached devices for my virtual machines.

— Reply to this email directly, view it on GitHub https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3434158630, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV6ODXZ6CZ2HVGAK6S4XOTT3Y7UUZAVCNFSM6AAAAACJ57G23GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIMZUGE2TQNRTGA . You are receiving this because you authored the thread.Message ID: @.***>

steeldomejeff avatar Oct 22 '25 20:10 steeldomejeff

i see the disk type is set to "file", yet i dont see the big picture. Seems to be a qcow overlay for a ceph network device. I dont have such a setup , so its hard for me to reproduce..

The disk type is detected within this function which seems to fail to detect this specific configuration:

https://github.com/abbbi/virtnbdbackup/blob/master/libvirtnbdbackup/virt/client.py#L337

but after that, not sure which other enhancements have to be done to support this type of VM disk conifguration. running with debug uption might help.

abbbi avatar Oct 22 '25 21:10 abbbi

I have this setup in the lab... I'll investigate the client.py segment you sent me and report back with what I find.

Standby. Thank you Michael.

On Wed, Oct 22, 2025 at 5:01 PM Michael Ablassmeier < @.***> wrote:

abbbi left a comment (abbbi/virtnbdbackup#286) https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3434189325

i see the disk type is set to "file", yet i dont see the big picture. Seems to be a qcow overlay for a ceph network device. I dont have such a setup , so its hard for me to reproduce..

The disk type is detected within this function which seems to fail to detect this specific configuration:

https://github.com/abbbi/virtnbdbackup/blob/master/libvirtnbdbackup/virt/client.py#L337

but after that, not sure which other enhancements have to be done to support this type of VM disk conifguration.

— Reply to this email directly, view it on GitHub https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3434189325, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV6ODX456LHMJZY5BCKNAN33Y7WC3AVCNFSM6AAAAACJ57G23GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIMZUGE4DSMZSGU . You are receiving this because you authored the thread.Message ID: @.***>

steeldomejeff avatar Oct 22 '25 21:10 steeldomejeff

what is see is:

<disk type='file' device='disk'>

and the code does:

diskType = disk.get("type")
[..]
 elif diskType == "file":

which should match this case, but it doesnt for some reason ... Having the complete config would help,..

abbbi avatar Oct 22 '25 21:10 abbbi

i tried to craft an vm with this type of disk config but fail to define:

sudo virsh define vm7.xml
error: Failed to define domain from vm7.xml
error: XML error: missing data file store format

running on debian trixie ..

maybe, if you want incremental backups for ceph/rbd backed devices, you need to specify like described here:

https://github.com/abbbi/virtnbdbackup?tab=readme-ov-file#libvirt-versions--v10100-1

abbbi avatar Oct 22 '25 21:10 abbbi

Send XML via email as the webui isn't rendering this correctly for some reason:

virsh dumpxml baseos-kfdvsfxo baseos-kfdvsfxo 8534fe43-2346-443d-9f55-e88e045c40d2 <libosinfo:libosinfo xmlns:libosinfo=" http://libosinfo.org/xmlns/libvirt/domain/1.0"> <libosinfo:os id="http://rockylinux.org/rocky/9"/> </libosinfo:libosinfo> <cockpit_machines:data> <cockpit_machines:os_variant>rocky9</cockpit_machines:os_variant> </cockpit_machines:data> 4194304 <currentMemory unit='KiB'>4194304</currentMemory> 1 hvm /usr/share/edk2/ovmf/OVMF_CODE.secboot.fd <nvram template='/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd' templateFormat='raw' format='raw'>/var/lib/libvirt/qemu/nvram/baseos-kfdvsfxo_VARS.fd <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> /usr/libexec/qemu-kvm

/dev/urandom

On Wed, Oct 22, 2025 at 5:12 PM Michael Ablassmeier < @.***> wrote:

abbbi left a comment (abbbi/virtnbdbackup#286) https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3434218695

i tried to craft an vm with this type of disk config but fail to define:

error: XML error: missing data file store format running on debian trixie ..

— Reply to this email directly, view it on GitHub https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3434218695, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV6ODX225ZXT7IND24RMF533Y7XLZAVCNFSM6AAAAACJ57G23GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIMZUGIYTQNRZGU . You are receiving this because you authored the thread.Message ID: @.***>

steeldomejeff avatar Oct 22 '25 21:10 steeldomejeff

I have this setup and fully functional in my lab. Let me know if there is anything you need me to try/test to validate the process.

steeldomejeff avatar Oct 22 '25 22:10 steeldomejeff

ok, now i see the full picture, the disk is configured differently as in your first posts here:

https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3434172733

The disk is configured as type "network" disk:

<disk type='network' device='disk'>

there is currently no support for this in virtnbdbackup, and i dont have such a setup at hand (at least not with an ceph backend). Adding support for the ceph backend without having the setup or the need for it is something that is out of scope for me currently (w/o getting paid). That said, its not only the backup part that needs to be implemented but also the restore part, in this particular case.

While i could add support for this using the "--raw" option, the restore is not handled correctly. The backup is then handled like a raw file and restore will not map and write to the network devices /ceph endpoints directly.

You should better try to configure the disk with an qcow overlay image using the new data-file features, it would make more sense because it allows for complete feature set:

https://github.com/abbbi/virtnbdbackup?tab=readme-ov-file#libvirt-versions--v10100-1

and this setup should be working, because it basically doesn't matter what backend the real data is using.

You have to check the libvirt documentation how and if it is possble to configure a metadata image for ceph backed devices. Otherwise the vm disk data-file setting must point to an mapped RBD device locally (as in, /dev/rbdX, that is mapped on the host system).

According to the examples in the libvirt test repository, it works for NBD, so i think it might work for CEPH in a similar way:

https://github.com/libvirt/libvirt/blob/master/tests/qemuxmlconfdata/disk-qcow2-datafile-store.x86_64-latest.xml#L44

I think thats an better way to sorting out backup for this type of machines, instead of attempting to treat the disks as raw devices which will not be handled correctly during restore, and, overall, do not provide the featureset this utility was created for in the first place.

As with the latest commit, it at least handles these kind of situations gracefully and prints a proper error message.

abbbi avatar Oct 23 '25 07:10 abbbi

Good day Michael,

Totally understood. I have the current code modifications in-place and working for backups and restores. I will continue testing all edge cases and I'll share the changes once I have completed all the testing in the lab.

On Thu, Oct 23, 2025 at 3:51 AM Michael Ablassmeier < @.***> wrote:

abbbi left a comment (abbbi/virtnbdbackup#286) https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3435595161

ok, now i see the full picture, the disk is configured differently as in your first posts. The disk is configured as type "network" disk:

there is currently no support for this in virtnbdbackup, and i dont have such a setup at hand. Implementing this without having the setup or the need for it is something that is out of scope for me currently (w/o getting paid). That said, its not only the backup part that needs to be implemented but also the restore part.

You could try to configure the disk with an qcow overlay image using the new data-file features, maybe this works for you:

https://github.com/abbbi/virtnbdbackup?tab=readme-ov-file#libvirt-versions--v10100-1

— Reply to this email directly, view it on GitHub https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3435595161, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV6ODX3RYVC6Y5ZDOCFGVJT3ZCCHNAVCNFSM6AAAAACJ57G23GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIMZVGU4TKMJWGE . You are receiving this because you authored the thread.Message ID: @.***>

steeldomejeff avatar Oct 23 '25 17:10 steeldomejeff

Good day Michael.

Do you want me to post the code updates here or email you directly?

On Thu, Oct 23, 2025 at 3:51 AM Michael Ablassmeier < @.***> wrote:

abbbi left a comment (abbbi/virtnbdbackup#286) https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3435595161

ok, now i see the full picture, the disk is configured differently as in your first posts. The disk is configured as type "network" disk:

there is currently no support for this in virtnbdbackup, and i dont have such a setup at hand. Implementing this without having the setup or the need for it is something that is out of scope for me currently (w/o getting paid). That said, its not only the backup part that needs to be implemented but also the restore part.

You could try to configure the disk with an qcow overlay image using the new data-file features, maybe this works for you:

https://github.com/abbbi/virtnbdbackup?tab=readme-ov-file#libvirt-versions--v10100-1

— Reply to this email directly, view it on GitHub https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3435595161, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV6ODX3RYVC6Y5ZDOCFGVJT3ZCCHNAVCNFSM6AAAAACJ57G23GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIMZVGU4TKMJWGE . You are receiving this because you authored the thread.Message ID: @.***>

steeldomejeff avatar Oct 24 '25 23:10 steeldomejeff

either way, maybe you can prepare an pull request?

abbbi avatar Oct 25 '25 07:10 abbbi

Pull submitted #288

Usage details and log outputs:

#RBD BASED BACKUP EXAMPLE virtnbdbackup --start-domain
-U qemu+ssh://root@HOSTNAME/system
--nbd-ip HOSTNAME
--nbd-port 21594
--ssh-user root
-d [VMNAME]
-l copy --raw
-o [OUTPUT PATH]/[TIMESTAMP - 20251028T211004Z] [2025-10-29 11:12:11] INFO lib common - printVersion [main]: Version: 2.38 Arguments: /usr/bin/virtnbdbackup --start-domain -U qemu+ssh://root@labsystem0011-1/system --nbd-ip labsystem0011-1 --nbd-port 21594 --ssh-user root -d rbdtest-mcmifnkt -l copy --raw -o /labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z [2025-10-29 11:12:11] INFO root virtnbdbackup - main [main]: Backup level: [copy] [2025-10-29 11:12:11] INFO virt client - _connect [main]: Connected to remote host: [labsystem0011-1] [2025-10-29 11:12:11] INFO root virtnbdbackup - main [main]: Libvirt library version: [10010000] [2025-10-29 11:12:11] INFO root virtnbdbackup - main [main]: NBD library version: [1.20.3] [2025-10-29 11:12:11] WARNING root virtnbdbackup - main [main]: Emulated TPM device attached: User action required. [2025-10-29 11:12:11] WARNING root virtnbdbackup - main [main]: Please manually backup contents of: [/var/lib/libvirt/swtpm/7d8b94fd-6d3e-442e-baef-a2a918822e63/] [2025-10-29 11:12:11] INFO root virtnbdbackup - main [main]: Backup will save [1] attached disks. [2025-10-29 11:12:11] INFO root virtnbdbackup - main [main]: Concurrent backup processes: [1] [2025-10-29 11:12:11] INFO ssh client - connect [main]: Connecting remote system [labsystem0011-1] via ssh, username: [root] [2025-10-29 11:12:11] INFO paramiko.transport transport - _log [Thread-1]: Connected (version 2.0, client OpenSSH_9.9) [2025-10-29 11:12:11] INFO paramiko.transport transport - _log [Thread-1]: Authentication (publickey) successful! [2025-10-29 11:12:11] INFO root virtnbdbackup - main [main]: Remote NBD Endpoint host: [labsystem0011-1] [2025-10-29 11:12:11] INFO root virtnbdbackup - main [main]: Temporary scratch file target directory: [/var/tmp] [2025-10-29 11:12:11] INFO root job - start [main]: Starting backup job. [2025-10-29 11:12:12] INFO fs fs - freeze [main]: Freezed [2] filesystems. [2025-10-29 11:12:12] INFO fs fs - thaw [main]: Thawed [2] filesystems. [2025-10-29 11:12:12] INFO nbd client - connect [vda]: Waiting until NBD server at [nbd://labsystem0011-1:21594/vda] is up. [2025-10-29 11:12:13] INFO nbd client - connect [vda]: Connection to NBD backend succeeded. [2025-10-29 11:12:13] INFO extenthandler extenthandler - queryBlockStatus [vda]: Start receiving backup extents. [2025-10-29 11:12:13] INFO extenthandler extenthandler - queryBlockStatus [vda]: Finished receiving extents. [2025-10-29 11:12:13] INFO root disk - backup [vda]: Got 47 extents to backup. [2025-10-29 11:12:13] INFO root disk - backup [vda]: 107374182400 bytes [100.0GiB] virtual disk size [2025-10-29 11:12:13] INFO root disk - backup [vda]: 26012024832 bytes [24.2GiB] of data extents to backup [2025-10-29 11:12:13] INFO root target - get [vda]: Write data to target file: [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/vda.copy.data.partial]. [2025-10-29 11:12:13] INFO root disk - backup [vda]: Creating full provisioned raw backup image [2025-10-29 11:16:21] INFO root disk - backup [vda]: RAW path: computing full-file Adler32 for /labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/vda.copy.data [2025-10-29 11:17:57] INFO root disk - backup [vda]: RAW path: Adler32=3120603567 [2025-10-29 11:17:57] INFO root virtnbdbackup - main [main]: Backup jobs finished, stopping backup task. [2025-10-29 11:17:57] INFO root metadata - backupConfig [main]: Saving VM config to: [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/vmconfig.copy.xml] [2025-10-29 11:17:57] INFO root metadata - backupBootConfig [main]: Save additional boot config [loader] to: [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/OVMF_CODE.secboot.fd] [2025-10-29 11:17:57] INFO ssh client - copyFrom [main]: Downloading file [/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd] to [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/OVMF_CODE.secboot.fd] [2025-10-29 11:17:57] INFO paramiko.transport.sftp sftp - _log [main]: [chan 0] Opened sftp connection (server version 3) [2025-10-29 11:17:57] INFO root metadata - backupBootConfig [main]: Save additional boot config [nvram] to: [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/rbdtest-mcmifnkt_VARS.fd] [2025-10-29 11:17:57] INFO ssh client - copyFrom [main]: Downloading file [/var/lib/libvirt/qemu/nvram/rbdtest-mcmifnkt_VARS.fd] to [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/rbdtest-mcmifnkt_VARS.fd] [2025-10-29 11:17:57] INFO paramiko.transport.sftp sftp - _log [main]: [chan 0] sftp session closed. [2025-10-29 11:17:57] INFO root virtnbdbackup - main [main]: Total saved disk data: [75.8GiB] [2025-10-29 11:17:57] INFO root virtnbdbackup - main [main]: Finished successfully

# RBD BASED VERIFY virtnbdrestore -i [BACKUP DIR PATH] -o verify -L [LOG OUTPUT FILE NAME] [2025-10-29 11:18:57] INFO lib common - printVersion [main]: Version: 2.38 Arguments: /bin/virtnbdrestore -i /labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z -o verify -L /tmp/virtnbdrestore-mztYgaKz5Y.log [2025-10-29 11:18:57] INFO root files - verify [main]: Computing checksum for: /labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/vda.copy.data [2025-10-29 11:21:25] INFO root files - verify [main]: Checksum result: 3120603567 [2025-10-29 11:21:25] INFO root files - verify [main]: Comparing checksum with stored information [2025-10-29 11:21:25] INFO root files - verify [main]: OK

# RBD RESTORE (SINGLE DISK-CREATE NEW RESTORED VM-LEAVE ORIGINAL UNTOUCHED) virtnbdrestore
-U qemu+ssh://root@HOSTNAME/system
-i [BACKUP DIR PATH]
-d [DISK NAME - e.g. vda]
-o rbd:[RBD POOL NAME]/[RBD IMAGE NAME]
--raw --adjust-config --auto-register --name [NEW VM NAME TO ATTACH TO] [2025-10-29 12:11:34] INFO lib common - printVersion [main]: Version: 2.38 Arguments: /usr/bin/virtnbdrestore -U qemu+ssh://root@labsystem0011-1/system -i /labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z -d vda -o rbd:auxiliaryvolumes.data/restored-vm-01 --raw --adjust-config --auto-register --name restored-vm-01 [2025-10-29 12:11:34] INFO virt client - _connect [main]: Connected to remote host: [labsystem0011-1] [2025-10-29 12:11:34] INFO root virtnbdrestore - main [main]: Using config file: [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/vmconfig.copy.xml] [2025-10-29 12:11:34] INFO root disk - restore [main]: Disk [vda]: restore target is Ceph RBD [rbd:auxiliaryvolumes.data/restored-vm-01] [2025-10-29 12:11:34] INFO root disk - restore [main]: Converting flat raw [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/vda.copy.data] -> [rbd:auxiliaryvolumes.data/restored-vm-01] via qemu-img convert (raw). [2025-10-29 12:17:21] INFO root vmconfig - removeUuid [main]: Removing uuid setting from vm config. [2025-10-29 12:17:21] INFO root vmconfig - setVMName [main]: Set name from [rbdtest-mcmifnkt] to [restored-vm-01] [2025-10-29 12:17:21] INFO root files - restore [main]: File [/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd]: for boot option [loader] already exists, skipping. [2025-10-29 12:17:21] INFO root files - restore [main]: File [/var/lib/libvirt/qemu/nvram/rbdtest-mcmifnkt_VARS.fd]: for boot option [nvram] already exists, skipping. [2025-10-29 12:17:21] INFO virt client - defineDomain [main]: Redefining domain based on adjusted config. [2025-10-29 12:17:21] INFO virt client - defineDomain [main]: Successfully redefined domain [restored-vm-01] [2025-10-29 12:17:21] INFO root virtnbdrestore - main [main]: Domain defined from adjusted config (RBD target). [2025-10-29 12:17:21] INFO root virtnbdrestore - main [main]: Saved adjusted VM XML to [/var/tmp/virtnbdrestore.vmconfig.adjusted.xml].

# RBD RESTORE (ALL DISKS-CREATE NEW RESTORED VM-LEAVE ORIGINAL UNTOUCHED) virtnbdrestore
-U qemu+ssh://root@HOSTNAME/system
-i [BACKUP DIR PATH]
-o rbd:[RBD POOL NAME]/[RBD IMAGE NAME]
--raw --adjust-config --auto-register --name [NEW VM NAME TO ATTACH TO]

# RBD BASED RESTORE (OVERWRITE-VDA DISK ONLY-WILL SHUTDOWN VM IF IT ALREADY EXISTS AND IS RUNNING) virtnbdrestore
-U qemu+ssh://root@HOSTNAME/system
-i [BACKUP DIR PATH]
-d [DISK NAME - e.g. vda]
-o rbd:[ORIGINAL RBD POOL NAME]/[ORIGINAL RBD IMAGE NAME]
--raw --adjust-config -D --auto-register --name [ORIGINAL VM NAME TO ATTACH TO]

# RBD BASED RESTORE (OVERWRITE-ALL DISKS-WILL SHUTDOWN VM IF IT ALREADY EXISTS AND IS RUNNING) virtnbdrestore
-U qemu+ssh://root@HOSTNAME/system
-i [BACKUP DIR PATH]
-o rbd:[ORIGINAL RBD POOL NAME]/[ORIGINAL RBD IMAGE NAME]
--raw --adjust-config -D --auto-register --name [ORIGINAL VM NAME TO ATTACH TO] [2025-10-29 12:26:02] INFO lib common - printVersion [main]: Version: 2.38 Arguments: /usr/bin/virtnbdrestore -U qemu+ssh://root@labsystem0011-1/system -i /labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z -o rbd:auxiliaryvolumes.data/rbdtest-mcmifnkt-os --raw --adjust-config -D --auto-register --name rbdtest-mcmifnkt [2025-10-29 12:26:02] INFO virt client - _connect [main]: Connected to remote host: [labsystem0011-1] [2025-10-29 12:26:02] INFO root virtnbdrestore - main [main]: Using config file: [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/vmconfig.copy.xml] [2025-10-29 12:26:02] INFO root disk - restore [main]: Disk [vda]: restore target is Ceph RBD [rbd:auxiliaryvolumes.data/rbdtest-mcmifnkt-os] [2025-10-29 12:26:02] INFO root disk - restore [main]: In-place overwrite detected for disk [vda] (auxiliaryvolumes.data/rbdtest-mcmifnkt-os); ensuring domain [rbdtest-mcmifnkt] is stopped before restore. [2025-10-29 12:26:02] INFO virt client - ensureDomainStopped [main]: Requesting graceful shutdown of domain [rbdtest-mcmifnkt]... [2025-10-29 12:26:10] INFO virt client - ensureDomainStopped [main]: Domain [rbdtest-mcmifnkt] shut down gracefully. [2025-10-29 12:26:10] INFO root disk - restore [main]: Converting flat raw [/labsystem001/clustermounts/machines/backup/a5c04a64-83fd-4a5d-a744-52a518de2929/2025-10/7d8b94fd-6d3e-442e-baef-a2a918822e63/20251029T151211Z/vda.copy.data] -> [rbd:auxiliaryvolumes.data/rbdtest-mcmifnkt-os] via qemu-img convert (raw). [2025-10-29 12:26:10] INFO libvirtnbdbackup.restore.disk disk - _qemu_img_convert_raw_to_rbd [main]: Target RBD exists; using in-place overwrite (-n). [2025-10-29 12:30:45] INFO root vmconfig - removeUuid [main]: Removing uuid setting from vm config. [2025-10-29 12:30:45] INFO root vmconfig - setVMName [main]: Set name from [rbdtest-mcmifnkt] to [rbdtest-mcmifnkt] [2025-10-29 12:30:45] INFO root files - restore [main]: File [/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd]: for boot option [loader] already exists, skipping. [2025-10-29 12:30:45] INFO root files - restore [main]: Restoring configured file [/var/lib/libvirt/qemu/nvram/rbdtest-mcmifnkt_VARS.fd] for boot option [nvram] [2025-10-29 12:30:45] WARNING virt client - defineDomain [main]: Domain [rbdtest-mcmifnkt] already exists; skipping re-definition because --auto-register is enabled. [2025-10-29 12:30:45] INFO root virtnbdrestore - main [main]: Domain defined from adjusted config (RBD target). [2025-10-29 12:30:45] INFO root virtnbdrestore - main [main]: Saved adjusted VM XML to [/var/tmp/virtnbdrestore.vmconfig.adjusted.xml].

steeldomejeff avatar Oct 29 '25 16:10 steeldomejeff

wow, thats lots of changes for this feature .. not yet sure if im going to merge.. maybe it makes sense you run on your own branch for a while if you need this feature. thanks for the PR

abbbi avatar Oct 30 '25 07:10 abbbi

It's a significant mod. Most of the major changes should be drop-ins. It provides a good working baseline for supporting RDP if desirable in the long-term goal.

steeldomejeff avatar Oct 30 '25 13:10 steeldomejeff

have you actually tried the approach with the data-file/datastore configuration in libvirt? It would be interesting if this would work, too. Then no changes are required for backup.

abbbi avatar Oct 30 '25 14:10 abbbi

I haven’t. I will investigate.

On Thu, Oct 30, 2025 at 10:14 AM Michael Ablassmeier < @.***> wrote:

abbbi left a comment (abbbi/virtnbdbackup#286) https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3468238889

have you actually tried the approach with the data-file/datastore configuration in libvirt? It would be interesting if this would work, too. Then no changes are required for backup.

— Reply to this email directly, view it on GitHub https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3468238889, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV6ODX6ZDM5PM6XDCPLKZIT32IMLBAVCNFSM6AAAAACJ57G23GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTINRYGIZTQOBYHE . You are receiving this because you authored the thread.Message ID: @.***>

steeldomejeff avatar Oct 30 '25 14:10 steeldomejeff

I just created a libvirt rbd pool and it appears it forms the same xml configuration in the VM that adding an rbd volume directly to the VM config does. In the XML config there is no reference to the libvirt pool name at all, only the ceph rbd pool and the monitors.

Due to this, I believe, we would still need the modified code to handle rbd images correctly.

On Thu, Oct 30, 2025 at 10:14 AM Michael Ablassmeier < @.***> wrote:

abbbi left a comment (abbbi/virtnbdbackup#286) https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3468238889

have you actually tried the approach with the data-file/datastore configuration in libvirt? It would be interesting if this would work, too. Then no changes are required for backup.

— Reply to this email directly, view it on GitHub https://github.com/abbbi/virtnbdbackup/issues/286#issuecomment-3468238889, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV6ODX6ZDM5PM6XDCPLKZIT32IMLBAVCNFSM6AAAAACJ57G23GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTINRYGIZTQOBYHE . You are receiving this because you authored the thread.Message ID: @.***>

steeldomejeff avatar Oct 30 '25 17:10 steeldomejeff

something like this, just adjusted to ceph should work:

    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/metadata.qcow2'>
        <dataStore type='network'>
          <format type='raw'/>
          <source protocol='nbd' name='Volume2/ImageDataFile'>
            <host transport='unix' socket='/path/to/sock/datafile'/>
          </source>
        </dataStore>
      </source>

like:

    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/metadata.qcow2'>
        <dataStore type='network'>
          <format type='raw'/>

      <auth username='admin'>
        <secret type='ceph' uuid='ae9a5d54-4c6f-4b4d-a50a-3a1be1a7a1f3'/>
      </auth>
      <source protocol='rbd' name='auxiliaryvolumes.data/baseos-kfdvsfxo-os'>
        <host name='192.168.59.102' port='3300'/>
        <host name='192.168.59.101' port='3300'/>
        <host name='192.168.59.103' port='3300'/>
       </dataStore>
      </source>

this way the image /var/lib/libvirt/images/metadata.qcow2 would be used as metadata image for checkpoints and bitmaps and the real data exposed by the NBD endpoint comes from the ceph source.

abbbi avatar Oct 30 '25 17:10 abbbi

or as alternative pointing to an RBD device directly:

     <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none' io='native' discard='unmap'/>
      <source file='/var/lib/libvirt/metadata.qcow2'>
        <dataStore type='file'>
          <format type='raw'/>
          <source file='/dev/rbd0'/>
        </dataStore>
      </source>
      <target dev='vda' bus='virtio'/>
    </disk>

abbbi avatar Oct 30 '25 17:10 abbbi