Allow Fleet Server to be run without TLS.

[naemono]

closes #6000

This potential change allows Fleet Server, running as Elastic Agent, to be run without TLS for such scenarios as running within a service mesh which could provide mTLS between applications/services, similar to how we allow Elasticsearch, and Kibana to run without TLS.

This additionally solves a panic when currently attempting to disable TLS on Fleet server:

[elastic-operator-0] {"log.level":"info","@timestamp":"2022-09-14T20:10:55.031Z","log.logger":"manager.eck-operator","message":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","service.version":"2.4.0+96282ca9","service.type":"eck","ecs.version":"1.4.0","controller":"agent-controller","object":{"name":"eck-fleet-eck-fleet-server","namespace":"istio-enabled"},"namespace":"istio-enabled","name":"eck-fleet-eck-fleet-server","reconcileID":"c0330ea8-bda6-48d5-8552-d25327c3959e"}
[elastic-operator-0] panic: runtime error: invalid memory address or nil pointer dereference [recovered]
[elastic-operator-0] 	panic: runtime error: invalid memory address or nil pointer dereference
[elastic-operator-0] [signal SIGSEGV: segmentation violation code=0x1 addr=0x120 pc=0x1769409]
[elastic-operator-0]
[elastic-operator-0] goroutine 2739 [running]:
[elastic-operator-0] sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
[elastic-operator-0] 	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:118 +0x1f4
[elastic-operator-0] panic({0x1a81660, 0x2de94a0})
[elastic-operator-0] 	/usr/local/go/src/runtime/panic.go:838 +0x207
[elastic-operator-0] github.com/elastic/cloud-on-k8s/v2/pkg/controller/agent.internalReconcile({{0x2017448, 0xc00180b110}, {0x201e198, 0xc00020c820}, {0x2016768, 0xc0007c2b80}, {0xc0007bff20, 0xc0007bff80, 0xc0007da000, 0xc0007da060}, ...})
[elastic-operator-0] 	/go/src/github.com/elastic/cloud-on-k8s/pkg/controller/agent/driver.go:130 +0xe89
[elastic-operator-0] github.com/elastic/cloud-on-k8s/v2/pkg/controller/agent.(*ReconcileAgent).doReconcile(_, {_, _}, {{{0x18b3b85, 0x5}, {0xc0017f21a0, 0x1d}}, {{0xc0006eb800, 0x1a}, {0x0, ...}, ...}, ...})
[elastic-operator-0] 	/go/src/github.com/elastic/cloud-on-k8s/pkg/controller/agent/controller.go:181 +0x585
[elastic-operator-0] github.com/elastic/cloud-on-k8s/v2/pkg/controller/agent.(*ReconcileAgent).Reconcile(0xc000772f20, {0x2017448?, 0xc00180b020?}, {{{0xc00076ebb0?, 0x10?}, {0xc000213500?, 0x40d927?}}})
[elastic-operator-0] 	/go/src/github.com/elastic/cloud-on-k8s/pkg/controller/agent/controller.go:147 +0x478
[elastic-operator-0] sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x20173a0?, {0x2017448?, 0xc00180b020?}, {{{0xc00076ebb0?, 0x1c017c0?}, {0xc000213500?, 0x4041f4?}}})
[elastic-operator-0] 	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121 +0xc8
[elastic-operator-0] sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0004a2d20, {0x20173a0, 0xc000634a80}, {0x1b01ee0?, 0xc0004d1be0?})
[elastic-operator-0] 	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320 +0x33c
[elastic-operator-0] sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0004a2d20, {0x20173a0, 0xc000634a80})
[elastic-operator-0] 	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273 +0x1d9
[elastic-operator-0] sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
[elastic-operator-0] 	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234 +0x85
[elastic-operator-0] created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
[elastic-operator-0] 	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:230 +0x325
^C[1]    43051 terminated  kubetail -n elastic-system -l app.kubernetes.io/instance=elastic-operator  1m

Also, an e2e test was added to ensure the full functionality all the way back to data within Elasticsearch.

[naemono]

run/e2e-tests

[naemono]

note. Need to test this against an older version of the stack to ensure that this works against older version, or restrict this to newer versions of the stack if not functional against older versions.

[CheyenneForbes]

+1 for this functionality