Optional annotations for the chao deployment and extra annotations for the daemonset

[kempsterc]

We would like to be able to provide these annotations. 🔈

In our lower environments we have spot nodes with spotless. The spot nodes apps/deployments are candidates for eviction (Preempted). We normally set the priority class high enough to stop this happening on gremlin and other tooling.

Since moving to bottlerocket we also introduced Karpenter. Some info here

Karpenter can disrupt pods when configuring node rotations, which means that the chao deployment or daemonset agents can be stopped (disrupted) mid test run.

We can apply annotations to stop this happening where we apply it. The chart annotation:

• karpenter.sh/do-not-disrupt: "true"

Not looking to get this merged but I wanted to do it to give context and make sure it works, and move to a feature request. Or commit review suggested changes and look to merge 😄

Testing: - helm 3

helm lint ✅ helm template with default values ✅ helm template with just chao annotations ✅ helm template with just gremlin extra annotations ✅ helm template with both additional annotations ✅ helm template -n gremlin gremlin . -f values.yaml | kubectl apply -f - ✅ Check pods are running ✅ Check daemon has started ✅ Run attack from gremlin ui ✅ Attack executed from start to finish ✅