feat(controller): retry strategy support on daemon containers, fixes #13705

[MenD32]

Addresses https://github.com/argoproj/argo-workflows/issues/13705 And https://github.com/argoproj/argo-workflows/issues/2963

Motivation

Add retryStrategy support to daemon container templates.

Some use cases require argo workflows features that aren't supported in resource template, f.e.

• readiness probe for the resource (resource might be running, but my expected condition
• pod logs, statuses, and fails in the UI
• passing input artifacts

Modifications

execution functions now consider "succeeded" daemoned nodes as failed

When a daemoned container completes execution, it is considered as failed, if it has a retry strategy it will retry.

the IP change in the node will cascade down to future executions.

Verification

added an examples to test locally

• examples/dag-daemon-retry-strategy.yaml
• examples/steps-daemon-retry-strategy.yaml

I simulated daemon failures by deleting the daemon pod manually

Beyond this PR

Thank you for submitting this! Have you ever thought of becoming a Reviewer or Approver on the project?

Argo Workflows is seeking more community involvement and ultimately more Reviewers and Approvers to help keep it viable. See Sustainability Effort for more information.

-->