fix: Warn when signal is handled

[holmanb]

fix: Warn when a signal is handled

Cloud-init is expected to run to completion. It does not expect to be
stopped or restarted. Therefore, cloud-init should loudly warn when this
happens so that the user is aware that the unexpected has happened. 

Problems:

- When systemd stops cloud-init, no warning is logged.
- When systemd restarts cloud-init, status info and return code from the
  current stage is lost.

The net effect of these problems is that when cloud-init is restarted
no warning is logged, and cloud-init status indicates no problem.

Communicate unexpected state by:

- Increase signal handling log level to ERROR, add log to stderr.
- Write status.json before exiting when sys.exit() is called.
- Warn when status.json contains existing data when the stage
  starts[1].
- Append existing errors and recoverable status.json errors rather
  than overwriting[2].

This should make cloud-init properly warn in status output and logs that
something is amiss.

[1] This should never happen and indicates that something restarted the
    service. When cloud-init doesn't have the opportunity to handle a
    signal (SIGKILL), other mitigations will fail.
[2] Pre-existing warnings were previously silently lost from
    `cloud-init status`'s output.

Fixes GH-5190

Fixes https://github.com/canonical/cloud-init/issues/5190

[CalvoM]

Is there a way of reproducing the issue, so that testing the difference becomes easy?

[TheRealFalcon]

@CalvoM , I think you can do something like systemctl restart cloud-final.service while cloud-final service is running. Otherwise, kill -5 the cloud-init process while it is running.

[holmanb]

@TheRealFalcon I just pushed an update. Is that along the lines of what you had in mind?

Unittests are failing, but if this is what you were thinking I'll fix up the tests prior to merge.

[holmanb]

@TheRealFalcon I've addressed your remaining comments. Please let me know what you think of my comments regarding re-raising.