Support RuntimeSDK in clusterctl alpha topology dryrun

[sbueringer]

As of today clusterctl alpha topology dryrun does not support RuntimeSDK (i.e. lifecycle and topology mutation hooks).

Goal of this issue is to implement support for RuntimeSDK in clusterctl to make it possible to dryrun ClusterClasses which use external patches.

In my opinion we should only implement support for the Topology Mutation hooks. Main goal of topology dryrun is to dryrun changes the controller would make. I think for this it's not necessary to also call lifecycle hooks during topology dryrun.

Some notes:

• This either requires supporting feature gates in clusterctl or hard-coding that the RuntimeSDK feature gate is enabled

/kind feature

[sbueringer]

/cc @ykakarap

[ykakarap]

/assign

[fabriziopandini]

/triage accepted

[fabriziopandini]

+1 to implement support for testing Topology Mutation hooks only

We briefly discussed possible options and as a first iteration we are going to implement the support for the following scenario - extensions must be installed in the Cluster and the corresponding ExtensionConfig should be in place (we do not accept ExtensionConfig in the file)

The implementation will be based on the following steps: - create a local registry leveraging on the status of the extension config already existing in the Cluster (vs doing an actual call to discovery) - identify required extensions by parsing CC in scope - open a kubectl proxy tunnel only to the extensions we actually need - update the local registry with the proxy address (to be used by clusterctl instead of the in-cluster address) - run topology plan with the local registry

[ykakarap]

I am happy to work on this once I have some bandwidth but if someone wants to work on this in the meantime I am happy to pair and help :)

[sbueringer]

Sounds good. One detail:

• open a kubectl proxy tunnel only to the extensions we actually need
• update the local registry with the proxy address (to be used by clusterctl instead of the in-cluster address)

Could be easier to modify the client to send the request via the kube-apiserver via proxy instead of opening a tunnel (/ port-forward (?)) per extension. Request would go roughly against: http://<apiserver-ip>/api/v1/namespaces/<extension-namespace>/services/<service-port>:<extension-service>:/proxy/<relative-path> (with the usual credentials) (not 100% sure about the URL format)

Note: We used this a while back to scrape metrics with Prometheus via the apiserver in other clusters, worked flawlessly

I assume we do the port-forward stuff only when a service is configured. If a URL is configured we would just use it directly.

[fabriziopandini]

Makes sense for me to explore also to think about modifying the client, this could remove from us the complexity of opening/closing tunnels cc @ykakarap

[ykakarap]

Sounds like an interesting option to explore.

Note: We used this a while back to scrape metrics with Prometheus via the apiserver in other clusters, worked flawlessly

Do you have a reference that I can take a look at?

[sbueringer]

I don't have access anymore unfortunately. Basically we told Promtheus via it's discovery mechanism to pull from a specific URL (~ the one I mentioned above) and use client certificates for authentication.

I'll try to dig up a bit more information.

[sbueringer]

I got it to work with kubectl. The following is a discovery request against the RuntimeExtension service with:

• namespace "default"
• name: "webhook-service"
• port "443"
• protocol "https" (the one deployed in our Tilt env)

kubectl create --raw /api/v1/namespaces/default/services/https:webhook-service:443/proxy/hooks.runtime.cluster.x-k8s.io/v1alpha1/discovery --v=8 -f <(echo '{"kind":"DiscoveryRequest","apiVersion":"hooks.runtime.cluster.x-k8s.io/v1alpha1"}')

I think if we track this back to how kubectl does it we should be able to do the same with client-go, but I don't want to spoil al the fun :)

(let me know if you want to pair on figuring this out, happy to)

[ykakarap]

Thanks for this find Stefan. Never know about this.

I don't want to spoil al the fun :)

haha.. thank you 🙏🏼 😃

[chrischdi]

It's easy to experiment using kubectl proxy and then using curl -X [GET/POST/...] 127.0.0.1:8001/..., especially when other requests than GET are required 😄

[fabriziopandini]

dropping from the milestone, we are late in implementation to make it happen in 1.3

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[fabriziopandini]

/lifecycle frozen

[fabriziopandini]

I'm going to close this issue now given that the GAP from this feature and what ClusterClass can do keeps growing and it doesn't seem there is much interest in contributing to this feature /close

[k8s-ci-robot]

@fabriziopandini: Closing this issue.

In response to this:

I'm going to close this issue now given that the GAP from this feature and what ClusterClass can do keeps growing and it doesn't seem there is much interest in contributing to this feature /close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.