Terminate running nodes after exiting `podTemplate` block

[Vlatombe]

This prevents orphan agents left behind after removing a dynamic pod template. In some cases where Jenkins is restarted just after exiting a podTemplate block, agents would lose their reference to pod template and cause various exceptions such as https://github.com/jenkinsci/kubernetes-plugin/pull/1045

• [X] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• [X] Ensure that the pull request title represents the desired changelog entry
• [X] Please describe what you did
• [ ] Link to relevant issues in GitHub or Jira
• [ ] Link to relevant pull requests, esp. upstream and downstream changes
• [ ] Ensure you have provided tests - that demonstrates feature works or fixes the issue

[jglick]

Seems reasonable. Doubt there would be any poor interactions with #1083 since this happens after node(POD_LABEL) {…} has exited.

[jglick]

KubernetesProvisioningLimits.unregister: Pod template count for runInPod-jrsz5 went below zero.

seems like a genuine regression.

[Dohbedoh]

Also wondering if maybe this requires some Queue.lock like retention strategies do. For example OnceRetentionStrategy.java.

[jglick]

https://github.com/jenkinsci/durable-task-plugin/pull/2#issuecomment-69762498 :shrug:

[Vlatombe]

Right, this probably due to the required queue lock...