Implement kubelet CredentialProvider

[tjamet]

Context

We currently suport 2 authentication sources. Docker and containerD.

Problems

1. Kubelet has a third one: CredentialProviderConfig

To be consistent, noe needs to implement all of those to be able to lookup the values in the relevant place.

2. The current containerD authentication is mixed with the Docker one (file and image pull secret ones).

Goal

Add supoort for kubelet CredentialProvider

Committer details

Local-Branch: HEAD

[dackroyd]

Need assistance here? My team is looking to use noe with images sourced from ECR, and this looks preferable to adding ECR specific support

[Fsero]

@dackroyd i think this one only needs fixing conflicts to be good to merge, i'd take a look during the week :)

[dackroyd]

@Fsero I got this running today in minikube, with the ECR credential provider. Ran into issues there with credentials not being provided due to there being some informational output on stderr from the provider, in addition to the token on stdout. Even if the call is a success, if anything is output to stderr, it’s being considered a failure.

Also ran into some other issues with the helm chart, specifically (the existing) flags being specified incorrectly so not taking effect.

[dackroyd]

Key log entry for the ECR provider is this one:

https://github.com/kubernetes/cloud-provider-aws/blob/master/cmd/ecr-credential-provider/main.go#L129

func (e *ecrPlugin) getPrivateCredsData(imageHost string, image string) (*credsData, error) {
	klog.Infof("Getting creds for private image %s", image)

Example output for that on stderr:

I0509 00:14:28.190934      72 main.go:124] Getting creds for private image {image}

In kubelet_credentials.go:

	err = execCommandOutput(ctx, &stdin, &stdout, &stderr, kubeToExec(provider.Env), filepath.Join(r.BinDir, provider.Name), provider.Args...)
	if err != nil {
		log.DefaultLogger.WithContext(ctx).WithError(err).Error("Could not execute kubelet credentials provider, skipping it")
		return
	}
	if stderr.Len() > 0 {
		log.DefaultLogger.WithContext(ctx).WithField("error", "stderr").Warn(stderr.String())
		return
	}

^ Anything output to stderr will cause the process to fail

[tjamet]

Hi @dackroyd . Thank you for your detailed feedback. I have changed how we handle stderr outputs so it does not fail any longer. I would expect to be able to merge this in the coming days after some further testing