Using the Helm SDK when no kubeconfig is available

[Sh4d1]

Currently when using helm SDK, a RESTClientGetter is needed. This is trivial when a kubeconfig file is available. However when the program only have access to the CA, token/certs and url in memory, it becomes a bit more complicated.

In fact the program needs to implement the following interfaces:

• ToRESTConfig() (*rest.Config, error)
• ToDiscoveryClient() (discovery.CachedDiscoveryInterface, error)
• ToRESTMapper() (meta.RESTMapper, error)

Of course it can still be done with some magic, but then it is also needed to create a new abstraction over the namespace with the ToRawKubeConfigLoader() method.

A fix would be to add a Namespace field to the kube client in order to lighten a bit the custom implementation written to use Helm3 without a kubeconfig

[mmellison]

Likewise, most of our apps have a reference to *rest.Config but not much else. It's easy to implement all of the given interfaces except ToRawKubeConfigLoader.

I think simpler initialization would help reduce the barrier to entry on getting started with the SDK.

[bacongobbler]

How does this differ from #7135? If they are the same feature request, can you please close one to keep the conversation in one ticket? Thanks!

[Sh4d1]

@bacongobbler its not the same. #7135 is about the Namespace flag on the different actions. This one is about an easier use of the sdk when no kubeconfig is available

[mmellison]

Is there a reason this was closed? Unless I'm mistaken, I don't see how #7373 addresses this in any way?

I think the intention of this issue was that it requires a lot of boilerplate to set up the Helm SDK in an environment where a standard kubeconfig doesn't exist, and so it would be a huge benefit to have an easier way to bootstrap the SDK.

[bacongobbler]

Merging #7373 automatically closes this issue as @Sh4d1 mentioned “Fixes #7377 (kind of)”. GitHub checks for the presence of “closes” or “fixes” proceeded by an issue number to mark issues as automatically closed once the PR is merged.

Re-opening.

[bacongobbler]

Is anyone planning to work on this, or shall I close this one out as inactive? Haven't seen any activity on this ticket for months.

[mmellison]

We're still looking forward to improvements made in this area. Checking out the latest improvements made to the Helm SDK and upgrading to the latest 3.x version is near the top of our backlog.

I will report back soon.

[bacongobbler]

We're still looking forward to improvements made in this area.

There have been no changes to this particular area of the code, which is why I'm reaching out.

I am asking if anyone from the community has looked into submitting a pull request to help improve this area, or if anyone may be able provide more insight on how they worked around this in their own projects.

Marking this as "help wanted" to signify that we aren't looking into this, but we are looking for guidance from the community to help make improvements in this area of the code, either in the form of pull requests or documentation.

[lemonli]

@bacongobbler I am interested on this. I have used the helm SDK with token/cert as auth-info instead of kubeconfig in our project to manage multi-cluster helm. Please assign me?

[vadasambar]

I was looking into the codebase for the places where the RESTClientGetter interface is being used and if we could get rid of it altogether and convert it into rest.Config and use that everywhere in the client sdk code , like a native kubernetes go client (one you create using client go library) and possibly everywhere else as well.

Something like

• Helm cli reads kubeconfig -> Helm code converts it into rest.Config -> Passes it to the helm sdk
• Code using helm sdk creates/gets rest.Config -> Passes it to the helm sdk

Places where RESTClientGetter is used

Interface

type RESTClientGetter interface {
	// ToRESTConfig returns restconfig
	ToRESTConfig() (*rest.Config, error) // NO NEED
	// ToDiscoveryClient returns discovery client
	ToDiscoveryClient() (discovery.CachedDiscoveryInterface, error)
	// ToRESTMapper returns a restmapper
	ToRESTMapper() (meta.RESTMapper, error) // NOT USED ANYWHERE
	// ToRawKubeConfigLoader return kubeconfig loader as-is
	ToRawKubeConfigLoader() clientcmd.ClientConfig
}

ToDiscoveryClient()

https://github.com/helm/helm/blob/4f7b9fcb67424008a839845397838b491036ca61/pkg/action/action.go#L242

// helm/pkg/action/action.go:237
237: // capabilities builds a Capabilities from discovery information.
238: func (c *Configuration) getCapabilities() (*chartutil.Capabilities, error) {
239: 	if c.Capabilities != nil {
240: 		return c.Capabilities, nil
241: 	}
242: 	dc, err := c.RESTClientGetter.ToDiscoveryClient()
243: 	if err != nil {
244: 		return nil, errors.Wrap(err, "could not get Kubernetes discovery client")
245: 	}

https://github.com/helm/helm/blob/4f7b9fcb67424008a839845397838b491036ca61/pkg/action/install.go#L151

// helm/pkg/action/install.go:149
149: 		// Invalidate the local cache, since it will not have the new CRDs
150: 		// present.
151: 		discoveryClient, err := i.cfg.RESTClientGetter.ToDiscoveryClient()
152: 		if err != nil {
153: 			return err
154: 		}

We might need to implement a function that returns discoveryClient ourselves.

ToRawKubeConfigLoader()

https://github.com/helm/helm/blob/4f7b9fcb67424008a839845397838b491036ca61/pkg/cli/environment.go#L150

// helm/pkg/cli/environment.go:148
148: // Namespace gets the namespace from the configuration
149: func (s *EnvSettings) Namespace() string {
150: 	if ns, _, err := s.config.ToRawKubeConfigLoader().Namespace(); err == nil {
151: 		return ns
152: 	}
153: 	return "default"
154: }

https://github.com/helm/helm/blob/4f7b9fcb67424008a839845397838b491036ca61/pkg/kube/client.go#L134

// helm/pkg/kube/client.go:130
130: func (c *Client) namespace() string {
131: 	if c.Namespace != "" {
132: 		return c.Namespace
133: 	}
134: 	if ns, _, err := c.Factory.ToRawKubeConfigLoader().Namespace(); err == nil {
135: 		return ns
136: 	}
137: 	return v1.NamespaceDefault
138: }

If we could find a way to get the namespace through some other way (maybe store it in some struct or allow it to be passed as a parameter), we can get rid of ToRawKubeConfigLoader().

https://github.com/helm/helm/blob/4f7b9fcb67424008a839845397838b491036ca61/pkg/kube/factory.go#L30

// helm/pkg/kube/factory.go:28
28: type Factory interface {
29: 	// ToRawKubeConfigLoader return kubeconfig loader as-is
30: 	ToRawKubeConfigLoader() clientcmd.ClientConfig
31: 	// KubernetesClientSet gives you back an external clientset
32: 	KubernetesClientSet() (*kubernetes.Clientset, error)

Same here. If ToRawKubeConfigLoader is only being used for getting the namespace, we should be able to get rid of this method altogether (unless I am missing something).

From my limited knowledge of the codebase, I think if we make the above changes, we should be able to get rid of the RESTClientGetter interface and use the rest.Config instead.

Note

• If there are other places where the methods are being used, like if the RESTClientGetter is required for some library function which is outside Helm codebase, then we might have a problem.
• I have a limited understanding of the codebase and I might be completely wrong in some places.

I am interested in this issue as well @bacongobbler. Maybe @lemonli and I can work on this together.

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[mitar]

Unstale.

[pytimer]

I use the helm sdk in my program, now i want to use rest.Config not kubeconfig, i already implement RESTClientGetter interface. But ToDiscoveryClient() (discovery.CachedDiscoveryInterface, error) function return CachedDiscoveryInterface struct, i want not use cached, whether the helm sdk support discovery.DiscoveryInterface ?

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[mitar]

Unstale.

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[mitar]

Unstale.

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[mitar]

Unstale.

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[mitar]

Unstale.

[khayong]

Refer this

https://medium.com/analytics-vidhya/using-helm-with-amazon-eks-without-a-kubeconfig-733f44a31b1d

[Jont828]

In my case, we have the kubeconfig as a string output from a secret and we'd like to initialize the client without having to write the config to a file as an intermediate step.

@jackfrancis It looks like there's already some discussion about what we talked about.

[Jont828]

I was able to find a solution for my case where I have the kubeconfig as a string and want to initialize the client without writing to a file. I followed the Medium article and came up with this solution

settings := helmCli.New()

restConfig, err := clientcmd.RESTConfigFromKubeConfig([]byte(kubeconfig))
if err != nil {
	return nil, nil, err
}

namespace := "default"
actionConfig, err := GetActionConfig(ctx, namespace, restConfig)

where we have the function

func GetActionConfig(ctx context.Context, namespace string, config *rest.Config) (*helmAction.Configuration, error) {
	log := ctrl.LoggerFrom(ctx)
	logf := func(format string, v ...interface{}) {
		log.V(4).Info(fmt.Sprintf(format, v...))
	}

	actionConfig := new(helmAction.Configuration)
	cliConfig := genericclioptions.NewConfigFlags(false)
	cliConfig.APIServer = &config.Host
	cliConfig.BearerToken = &config.BearerToken
	cliConfig.Namespace = &namespace
	// Drop their rest.Config and just return inject own
	wrapper := func(*rest.Config) *rest.Config {
		return config
	}
	cliConfig.WithWrapConfigFn(wrapper)
	if err := actionConfig.Init(cliConfig, namespace, "secret", logf); err != nil {
		return nil, err
	}
	return actionConfig, nil
}

[pegasas]

hi, @bacongobbler, I have a plan to integrate client-go clientcmd client config into this, I have tested it work on azure kubernetes cluster.

type RESTClientGetter struct {
	Namespace    string
	ClientConfig clientcmd.ClientConfig
}

func NewRESTClientGetter(namespace string, clientConfig clientcmd.ClientConfig) *RESTClientGetter {
	return &RESTClientGetter{
		Namespace:    namespace,
		ClientConfig: clientConfig,
	}
}

func (c *RESTClientGetter) ToRESTConfig() (*rest.Config, error) {
	config, err := c.ClientConfig.ClientConfig()
	if err != nil {
		return nil, err
	}

	return config, nil
}

func (c *RESTClientGetter) ToDiscoveryClient() (discovery.CachedDiscoveryInterface, error) {
	config, err := c.ToRESTConfig()
	if err != nil {
		return nil, err
	}

	// The more groups you have, the more discovery requests you need to make.
	// given 25 groups (our groups + a few custom conf) with one-ish version each, discovery needs to make 50 requests
	// double it just so we don't end up here again for a while.  This config is only used for discovery.
	config.Burst = 100

	discoveryClient, _ := discovery.NewDiscoveryClientForConfig(config)
	return memory.NewMemCacheClient(discoveryClient), nil
}

func (c *RESTClientGetter) ToRESTMapper() (meta.RESTMapper, error) {
	discoveryClient, err := c.ToDiscoveryClient()
	if err != nil {
		return nil, err
	}

	mapper := restmapper.NewDeferredDiscoveryRESTMapper(discoveryClient)
	expander := restmapper.NewShortcutExpander(mapper, discoveryClient)
	return expander, nil
}

func (c *RESTClientGetter) ToRawKubeConfigLoader() clientcmd.ClientConfig {
	return c.ClientConfig
}

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[mitar]

Unstale.

[Segflow]

We are also waiting for a fix for this

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[mitar]

Unstale.