[enhancement]: Improve hotplug triggering to avoid unconfigured NICs
Enhancement
At the moment, hook-hotplug only fires if all cloud-init stages have finished, thus if a NIC is added/removed while cloud-init those NICs won't be handled by hotplug.
They may or may not be configured by init-local depending on the time when they are added and the status of updates.when on non-first boots.
This is racey and needs to be addressed.
One possible way to address this is to modify hook-hotplug to fire after init-local has run (because this stage may configure the NICs on first boot or subsequent boots if updates.when contains boot) making sure that cloud-init-hotplugd.service waits until the other cloud-init services have finished, to avoid conflicts read/writing the ds pickle.
More context:
https://github.com/canonical/cloud-init/pull/4799#issuecomment-1932008837 https://github.com/canonical/cloud-init/pull/4799#issuecomment-1932208094
Tracking in https://warthogs.atlassian.net/browse/SC-1716.