sops-encrypted manfiests built by controller give red herring error

[scott-grimes]

Issue Summary

When deploying a kustomization with sops decryption an error is thrown which makes it appear that a rendered value is invalid, when in fact the kustomization itself has an incorrectly configured service account which lacks permissions to create the object

Details

The error message Secret forbidden, error: data values must be of type string is thrown if you happen to create a kustomization which renders a sops-encrypted manifest and forgot to add the decryption.provider section (ala https://github.com/fluxcd/flux2/discussions/1999)

However, the exact same error message can be thrown even if your kustomization is configured correctly, your manifests are encrypted correctly, and the child manifest is decrypted and rendered correctly.

To Replicate

I've constructed a minimal example at https://github.com/scott-grimes/fluxErrorRepo

You can replicate it (flux version 0.31.1) by running the following

minikube start 
flux install
kustomize build https://github.com/scott-grimes/fluxErrorRepo/ | kubectl apply -f -
flux get ks

Or by downloading the repo and following the instructions there

The problem is that the kustomization specifies a serviceAccountName that exists, but lacks permissions to create the resources. The error message is unclear. Rather than displaying something like "permission denied" you see forbidden, error: data values must be of type string, which is the same message that would happen if there was a problem with the decryption.

[stefanprodan]

Yeah we hide all errors because the Kubernetes API includes the secret values in the error message. I know this is frustrating to debug but we did this to avoid leaking secrets.

[kingdonb]

That is annoying but very sensible, is there anything to be done about this issue or should we close it?

(I applied the label wontfix, but this also seems like a usability issue. There is no UX label here, so I also added the Docs label...)

[scott-grimes]

Hmm perhaps an --insecure-show-raw-errors flag to allow for further debugging?

In this case it would have allowed us to rule out a possible encryption problem and easily see it was a permission problem. I'm assuming the error message would have been something like "service account xxx cannot create resource xxx in namespace xxx" had we been able to see it.

There's bound to be enough problems that would fall into this class of errors for other people to find this useful?

On Thu, Jun 23, 2022, 1:59 PM Kingdon Barrett @.***> wrote:

That is annoying but very sensible, is there anything to be done about this issue or should we close it?

— Reply to this email directly, view it on GitHub https://github.com/fluxcd/kustomize-controller/issues/683#issuecomment-1164709525, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQZUACJMGHKWJNFN4772XTVQSQWXANCNFSM5Y7SEHPQ . You are receiving this because you authored the thread.Message ID: @.***>

[loekalive]

I think hiding the error is fine if that's what's needed to keep things safe. Please improve the actual error that is shown though. The error Secret forbidden, error: data values must be of type string suggests there's something wrong with the content of the Secret. Which is annoyingly different from the actual situation I was debugging: The flux user has no role bound that has the create verb for the secret resource.

[stefanprodan]

Fixed in Flux v2.2