[FR] s3gw-ui: allow specifying custom certificate authorities (CA) for self-signed certificates

[jecluis]

With https://github.com/aquarist-labs/s3gw-ui/pull/269 as a quick fix to https://github.com/aquarist-labs/s3gw/issues/737 , we are essentially disabling SSL certificate validation between the UI backend and the s3gw service.

This is less than ideal, because we are breaking the trust relationship we must ensure between the UI backend and the s3gw service.

Instead, we should ensure we have a way to provide the CA used to sign the self-signed certificates to the UI backend container, so we can ensure the validity of the certificates even if self-signed.

At this point it's not clear how this should be implemented.

In a kubernetes environment, we could have the CA being provided as a secret, and then magically used to validate the certificates. It will depend on how cert-manager does self-signed certificates, and whether it creates a CA. This also requires some research.

[jecluis]

This is mostly relevant for contexts outside of a LH deployment, because

1. LH controller deploys the s3gw-ui container in the same pod as the s3gw container, effectively allowing us to do inner-pod communication between the two
2. Assuming we get to have an ingress shared between LH and the various s3gw-ui instances, with the latter being subpaths in the longhorn UI URL, we get a free SSL certificate out of it (the one for the LH UI as a whole)

Hence,

For 1. we can ignore the SSL validity, we can even do plain HTTP, because the traffic should be limited and isolated to the pod. For 2. we don't need to care about the SSL certificate for the various s3gw-ui instances, leaving it up to the user to handle their own CA.

For these reasons, we are pushing this issue until after LH 1.6, and we'll reevaluate its priority around v0.25.0.