Terraform keeps creating a vm with domain type ="kvm" instead of "qemu"

[brokedba]

System Information

Linux distribution

Centos7

Terraform version

Terraform v0.12.24
+ provider.libvirt (unversioned)
+ provider.template v2.1.2

Provider and libvirt versions

[root@localhost plugins]# ./terraform-provider-libvirt -version
./terraform-provider-libvirt 0.6.2+git.1585292411.8cbe9ad0
Compiled against library: libvirt 4.1.0
Using library: libvirt 4.5.0
Running hypervisor: QEMU 1.5.3
Running against daemon: 4.5.0 

---

Description of Issue/Question

My terraform apply run works perfectly with no warnings/erros . 4 resources are created but the domain XML configuration file has the domain value to kvm instead of qemu which keeps the vm from starting in my environment . If I change the value manually to qemu the vm starts right. I haven't seen any attribute in the below tf that specifies the domain type . is there any ?

Setup

[root@localhost terraform]# cat libvirt.tf
provider "libvirt" {
 uri = "qemu:///system"
}

# We fetch the latest ubuntu release image from their mirrors
resource "libvirt_volume" "ubuntu-qcow2" {
 name   = "ubuntu-qcow2"
 pool   = "default" #CHANGE_ME
 source = "https://cloud-images.ubuntu.com/releases/xenial/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img"
 format = "qcow2"
}

# Create a network for our VMs
resource "libvirt_network" "terranet" {
 name      = "terranet"
 addresses = ["192.168.78.0/24"]
 autostart = true
 dhcp {
   enabled = true
 }
}

# Use CloudInit to add our ssh-key to the instance
resource "libvirt_cloudinit_disk" "commoninit" {
 name           = "commoninit.iso"
 pool           = "default" #CHANGEME
 user_data      = data.template_file.user_data.rendered
 network_config = data.template_file.network_config.rendered
}

data "template_file" "user_data" {
 template = file("${path.module}/cloud_init.cfg")
}

data "template_file" "network_config" {
 template = file("${path.module}/network_config.cfg")
}

# Create the machine
resource "libvirt_domain" "domain-ubuntu" {
 name   = "terraform-db2"
 memory = "512"
 vcpu   = 1

 cloudinit = libvirt_cloudinit_disk.commoninit.id

 network_interface {
   network_id   = libvirt_network.terranet.id
   network_name = "terranet"
 }

 # IMPORTANT
 # Ubuntu can hang is a isa-serial is not present at boot time.
 # If you find your CPU 100% and never is available this is why
 console {
   type        = "pty"
   target_port = "0"
   target_type = "serial"
 }

 console {
   type        = "pty"
   target_type = "virtio"
   target_port = "1"
 }

 disk {
   volume_id = libvirt_volume.ubuntu-qcow2.id
 }
 graphics {
   type        = "spice"
   listen_type = "address"
   autoport    = "true"
 }
}

Steps to Reproduce Issue

[root@localhost terraform]# terraform apply 
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.                                                                                                                                                        
[root@localhost terraform]# virsh start terraform-db2 

[root@localhost terraform]# virsh list --all                                                                                                                                                                          Id    Name                           State                                                                                                                                                                        ----------------------------------------------------     
-     terraform-db2                  shut off  

---

Additional information:

when I check the xml config file of the vm I see the below kvm as domain type

$virsh edit terraform-db2
<domain type='kvm'>
<name>terraform-db2</name> 
...                                                                                                                                                                                                   

when I change the domain to qemu the vm starts normally.

[root@localhost terraform]# virsh list
 Id    Name                           State
----------------------------------------------------
 13    terraform-db2                  running

Can Anybody tell me how can I make terraform provision a vm with the right domain-type (qemu instead of kvm) ?? Having to modify the file manually is pointless . thank you for reading me. Koss

[brokedba]

additional information : the host Is actually a virtualbox virtual machine with nested virtualization enabled . qemu virtualization works well under the feature as I created couple of vms already in case you were wondering .

[brokedba]

Found the solution(workaround) .

• As nested virtualization with virtualbox doesn't quite support kvm domain type yet it seems.

the goal was to make libvirt provider chose qemu instead of kvm while provisioning the resource. this issue is similar to what openstak and miniduke encounter when they use kvm within vbox "could not find capabilities for domaintype=kvm"

• with the help of @titogarrido we found out that the the logic inside terraform-provider-libvirt/libvirt/domain_def.go check implied kvm only supported virtualization.

• In case of nested environment qemu had to be specified for the vm to launch

• Therefore the workaround is to set the below variable

     export TERRAFORM_LIBVIRT_TEST_DOMAIN_TYPE="qemu" 
  

Below check happens then qemu is selected

if v := os.Getenv("TERRAFORM_LIBVIRT_TEST_DOMAIN_TYPE"); v != "" {
		domainDef.Type = v
	} else {
		domainDef.Type = "kvm"
	}

Note : it would be nice to include this variable inside terraform declaration instead of relying on an env variable.

[MalloZup]

@KoussHD Happy that you found your way but actually you found an artifact hack from the past ( at time I wasn not maintaining it)

Regarding the variable TERRAFORM_LIBVIRT_TEST_DOMAIN_TYPE I think that this variable is wrong and should removed. (it was used for running test inside travis). That variable was used for testing purposes and is more an hack then anything to me. Already the name and usage for testing right now collide with the fact that you aren't using to testing.

Moreover the name doesn't give any hint or any documentation is provided to what the variable goals behaviour try to achieve.

I think we could remove this env variable and create it inside the terraform declaration with an acceptable name which is not technical driven and documentation/spec which can include help the user to understand why this variable might be needed.

[brokedba]

I couldn't agree more sir. Looking forward to seeing the change implemented.

[captainwasabi]

I am having to use this too. The production system is bare metal, but the dev systems are virtualized (at least that's the plan) so they are creating nested VMs (it's dev so performance doesn't matter so much)

[sudhakso]

TERRAFORM_LIBVIRT_TEST_DOMAIN_TYPE

Solves the domain type issue, and its good to retain this hack for test purposes.

[mark-rushakoff]

I am running a qemu VM for x86_64 on an M1 Mac.

I can manually run my target Ubuntu image when launching qemu directly from the command line. With libvirt, when I have a resource like

resource "libvirt_domain" "controlplane" {
  count = var.project.local.control_plane_size
  name = "control-${count.index}"
  memory = "1024"

  arch = "x86_64"
  machine = "pc"

  disk {
    volume_id = libvirt_volume.controlplane[count.index].id
    scsi = true
  }
}

then terraform apply reports an error like:

error defining libvirt domain: unsupported configuration: Emulator '/opt/homebrew/bin/qemu-system-x86_64' does not support virt type 'kvm'

When I run with TERRAFORM_LIBVIRT_TEST_DOMAIN_TYPE=qemu terraform apply, I can successfully create the libvirt domain.

The environment variable solution is okay. I could possibly use an XSLT so that consumers of this project don't have to set an environment variable. But overall I would much prefer I could just set type = "qemu" on the resource instead of depending on this environment variable.