terraform-provider-libvirt
terraform-provider-libvirt copied to clipboard
dmacvicar
CoreOS Ignition only works on first boot, so updates are no-ops
System Information
Linux distribution
5.3.1-arch1-1-ARCH #1 SMP PREEMPT
Terraform version
Terraform v0.12.9
Provider and libvirt versions
terraform-provider-libvirt was not built correctly
Compiled against library: libvirt 5.5.0
Using library: libvirt 5.5.0
Running hypervisor: QEMU 4.1.0
Running against daemon: 5.5.0
Git Commit:
3a11558c00af441cfe41e06896922ff7e5041f7f
Checklist
- [x] Is it a bug or something that does not work as expected? Please make sure you fill the version information below:
Description of Issue
After updating an ignition config & running terraform apply I notice the changes reflected on disk, but not within the VM itself.
Setup
https://gist.github.com/FreekingDean/dd3735523433f49f7484613107555761
Steps to Reproduce Issue
Not sure if logs are relevant, but can add if needed!
Additional information:
Nothing I can think to note!
I think here we just miss the update operation on the CRUD of terraform. This might not be trivial to implement or at least need some research.
thx for issue
@zeenix nope that version is a cosmetic thing (https://github.com/dmacvicar/terraform-provider-libvirt/blob/master/main.go#L17).
This come when users use RPM versions or other packages not build from source. I think the --version command might have other room of improvement there but the risk is that we over complicated things..
But for sure we could improve the errror msg :grin:
After updating an ignition config & running terraform apply I notice the changes reflected on disk, but not within the VM itself.
I don't understand this part. The ignition resource is supposed to help automating the packaging of the resource as a volume, and then passing that to QEMU via firmware config.
What else are you expecting it to do, or what are you not seeing, when you say "not within the VM itself"?
I think this is my misunderstanding of the coreos ignition setup: https://coreos.com/ignition/docs/latest/what-is-ignition.html#when-is-ignition-executed. It seems that changes to the coreos ignition system would never be reflected since it only happens on first boot. I wonder if its wise to add some documentation around this. For example heres how I am solving this: https://github.com/FreekingDean/tf-modules/blob/master/vm/main.tf#L67-L69
resource "libvirt_volume" "os" {
name = "os-qcow2_${var.orchistration_type}.${count.index}"
pool = libvirt_pool.pool.name
source = local.os
format = "qcow2"
count = var.node_count
depends_on = [
libvirt_ignition.ignition
]
}
FWIW, cloudinit seems like a better alternative as well, so I'll be migrating to that anyway. (Im going to re-title to be more accurate)
I think this is my misunderstanding of the coreos ignition setup: https://coreos.com/ignition/docs/latest/what-is-ignition.html#when-is-ignition-executed. It seems that changes to the coreos ignition system would never be reflected since it only happens on first boot. I wonder if its wise to add some documentation around this.
Yes. Ignition is comparable to what AutoYaST or Kickstart do. It is not a configuration management system.
For example heres how I am solving this: https://github.com/FreekingDean/tf-modules/blob/master/vm/main.tf#L67-L69
resource "libvirt_volume" "os" { name = "os-qcow2_${var.orchistration_type}.${count.index}" pool = libvirt_pool.pool.name source = local.os format = "qcow2" count = var.node_count depends_on = [ libvirt_ignition.ignition ] }
What is the advantage of making the os depend on the ignition data explicitly?
If I understand correctly, what you are doing is making the volume to be recreated if ignition data changes, which results in the profile to be reapplied, while in the default case, it is the VM which depends on the profile, which means the VM gets re-created on change, but not the volume (which is the only mutable and therefore changed resource by the ignition profile).
Do I understand correctly?
The way coreos ignition decides to run is based on a file in the coreos/first_boot directory. After it is run once it will not run again. It is also not an immutable series of functions, so I need a fresh "install" of coreos if I want ignition to successfully run again.
Makes sense.
Unfortunately, the ignition resource is attached to the VM, as it is the firmware configuration and the volume attachment that needs to be modified. That seems to skip the dependency between the ignition profile and the volume, which you very well do manually with a depends_on.
Do you see a better way of fixing this? Otherwise we could just add a note to the documentation.