tanzu-cli
tanzu-cli copied to clipboard
Published
20 hours ago •
vmware-tanzu
vmware-tanzu
Avoid uselessly trying to migrate plugins
What this PR does / why we need it
The only time we need to migrate context-scope plugins is after moving from a CLI < 1.3 to a newer one. This PR makes use of the existing global initializer to do this migration once, for all contexts.
This avoids an unnecessary slow down for every CLI commands, especially shell completion which should be as fast as possible. We can see an improvement of 30% to 40% in speed for basic shell completion commands.
Note:
- before this PR, context-scoped plugins would be migrated only for the active context; this means that when another context becomes active, it may also need to have its plugins migrated
- also before this PR, context-scope plugins that have been migrated will then always be shown as standalone plugins, even if their context is made inactive
- with this PR, since we only run the plugin migration once, we must migrate the context-scoped plugins for all contexts immediately. This is a slight change in behaviour because context-scoped plugins for an inactive (and possibly old context) will now appear as standalone plugins right away for the new CLI. Since this new behaviour is similar to point 2 above, I feel it is acceptable
Which issue(s) this PR fixes
Fixes N/A
Describe testing done for PR
Test with a CLI 1.2 and a CLI using this PR. We install context-scope plugins and make sure they are migrated as expected.
$ tanzu1.2 version
version: v1.2.0
buildDate: 2024-02-07
sha: f3abe62e
arch: arm64
$ tz version
version: v1.6.0-dev
buildDate: 2024-11-28
sha: 05e593e30
arch: arm64
# Install the context-scope plugins
$ tanzu1.2 context use tkg1
[i] Successfully activated context 'tkg1' of type 'kubernetes'.
[i] Checking for required plugins for context 'tkg1'...
[i] The following plugins will be installed for context 'tkg1' of contextType 'kubernetes':
NAME TARGET VERSION
cluster kubernetes v0.31.1
feature kubernetes v0.31.1
kubernetes-release kubernetes v0.31.1
[i] Installed plugin 'cluster:v0.31.1' with target 'kubernetes' (from cache)
[i] Installed plugin 'feature:v0.31.1' with target 'kubernetes' (from cache)
[i] Installed plugin 'kubernetes-release:v0.31.1' with target 'kubernetes' (from cache)
[i] Successfully installed all required plugins
# Confirm they are in the Catalog under the old serverPlugins section
$ grep -A4 serverPlugin ~/.cache/tanzu/catalog.yaml
serverPlugins:
tkg1:
cluster_kubernetes: /Users/kmarc/Library/Application Support/tanzu-cli/cluster/v0.31.1_9d63b407fc78ad86f00342ae920589b3e0eba4282a01b547e0cee9188f338a77_kubernetes
feature_kubernetes: /Users/kmarc/Library/Application Support/tanzu-cli/feature/v0.31.1_8c38d6db1c0cd2cd0b894cd46bbd4ffdb89e3b21df496f7df943aa57188780c7_kubernetes
kubernetes-release_kubernetes: /Users/kmarc/Library/Application Support/tanzu-cli/kubernetes-release/v0.31.1_b4c395c7b469c048968c57314ae901e84cbd6606222da227a50a84a8e2a8aa69_kubernetes
# Run a command of the newer CLI to see the global initializer run
# Notice also that the cluster, feature, kubernetes-release plugins are shown in the list
# with the recommended column as expected; this shows they have been migrated
$ tz plugin list
Some initialization of the CLI is required.
Let's set things up for you. This will just take a few seconds.
Refreshing the 27 installed plugins...
Initialization done!
==
NAME DESCRIPTION TARGET INSTALLED RECOMMENDED STATUS
apply Apply a resource file operations v0.6.5 installed
apps Applications on Kubernetes kubernetes v0.13.2 installed
appsv2 Applications on Tanzu Platform global v0.21.2-dirty installed
build Generate app image from source code global v0.16.0 installed
builder Build Tanzu components global v1.5.0 installed
cluster Kubernetes cluster operations kubernetes v0.31.1 v0.31.1 installed
cluster Plugin for operating clusters in the organization operations v0.11.6 installed
clustergroup A group of Kubernetes clusters operations v0.7.5 installed
domain tanzu-cli domain plugin global v0.1.0 installed
domain-binding tanzu-cli domain binding plugin global v0.1.4 installed
egress Provides crud operations for egresspoints global v0.0.5 installed
ekscluster ekscluster plugin is used to provision and manage the tmc eks clusters. operations v0.6.5 installed
feature Operate on features and featuregates kubernetes v0.31.1 v0.31.1 installed
kubectl-plugin Provides the kubectl functionality global v0.0.0 installed
kubernetes-release Kubernetes release operations kubernetes v0.31.1 v0.31.1 installed
kubetool Invokes any binary after setting KUBECONFIG to match the tanzu CLI global v0.0.0 installed
management-cluster A registered Management cluster operations v0.6.5 installed
package tanzu package management kubernetes v0.35.8 installed
policy Policy management for resources operations v0.6.5 installed
project Create, view, list, use and delete Tanzu projects. global v0.0.0-dev-7741e5e installed
provider-eks-cluster provider-eks-cluster plugin is used to manage and unmanage tmc provider eks operations v0.6.5 installed
clusters
rbac Tanzu Platform Role-Based Access Control global v0.0.0-dev-cde9be5 installed
resource manage resources in a Kubernetes cluster global v0.5.1 installed
secret Tanzu secret management kubernetes v0.33.5 installed
services Commands for working with services, classes and claims kubernetes v0.14.0 installed
space Tanzu space lifecycle management global v0.4.0 installed
telemetry configure cluster-wide settings for vmware tanzu telemetry global v1.1.0 installed
# Confirm the old serverPlugin section is gone
$ grep -A4 serverPlugin ~/.cache/tanzu/catalog.yaml
$
# Recreate the old serverPlugins section by reinstalling the context-scope
# plugins with the old CLI
$ tanzu1.2 context use tkg1
[i] Successfully activated context 'tkg1' of type 'kubernetes'.
[i] Checking for required plugins for context 'tkg1'...
[i] The following plugins will be installed for context 'tkg1' of contextType 'kubernetes':
NAME TARGET VERSION
cluster kubernetes v0.31.1
feature kubernetes v0.31.1
kubernetes-release kubernetes v0.31.1
[i] Installed plugin 'cluster:v0.31.1' with target 'kubernetes' (from cache)
[i] Installed plugin 'feature:v0.31.1' with target 'kubernetes' (from cache)
[i] Installed plugin 'kubernetes-release:v0.31.1' with target 'kubernetes' (from cache)
[i] Successfully installed all required plugins
$
$ grep -A4 serverPlugin ~/.cache/tanzu/catalog.yaml
serverPlugins:
tkg1:
cluster_kubernetes: /Users/kmarc/Library/Application Support/tanzu-cli/cluster/v0.31.1_9d63b407fc78ad86f00342ae920589b3e0eba4282a01b547e0cee9188f338a77_kubernetes
feature_kubernetes: /Users/kmarc/Library/Application Support/tanzu-cli/feature/v0.31.1_8c38d6db1c0cd2cd0b894cd46bbd4ffdb89e3b21df496f7df943aa57188780c7_kubernetes
kubernetes-release_kubernetes: /Users/kmarc/Library/Application Support/tanzu-cli/kubernetes-release/v0.31.1_b4c395c7b469c048968c57314ae901e84cbd6606222da227a50a84a8e2a8aa69_kubernetes
# Now uninstall the 3 context-scope plugins to get an empty serverPlugins section
$ tanzu1.2 plugin uninstall cluster -t k8s
Uninstalling plugin 'cluster' for target 'kubernetes'. Are you sure? [y/N]: y
[i] Uninstalling plugin 'cluster' for target 'kubernetes'
[i] Uninstalling plugin 'cluster' for target 'kubernetes'
[ok] successfully uninstalled plugin 'cluster'
$ tanzu1.2 plugin uninstall kubernetes-release -y -t k8s
[i] Uninstalling plugin 'kubernetes-release' for target 'kubernetes'
[i] Uninstalling plugin 'kubernetes-release' for target 'kubernetes'
[ok] successfully uninstalled plugin 'kubernetes-release'
$ tanzu1.2 plugin uninstall feature -y -t k8s
[i] Uninstalling plugin 'feature' for target 'kubernetes'
[i] Uninstalling plugin 'feature' for target 'kubernetes'
[ok] successfully uninstalled plugin 'feature'
$
$ grep -A4 serverPlugin ~/.cache/tanzu/catalog.yaml
serverPlugins:
tkg1: {}
# Use the new CLI and notice the cluster, feature, kubernetes-release plugins
# are listed as uninstalled as expected
$ tz plugin list
Some initialization of the CLI is required.
Let's set things up for you. This will just take a few seconds.
Refreshing the 24 installed plugins...
Initialization done!
==
NAME DESCRIPTION TARGET INSTALLED RECOMMENDED STATUS
apply Apply a resource file operations v0.6.5 installed
apps Applications on Kubernetes kubernetes v0.13.2 installed
appsv2 Applications on Tanzu Platform global v0.21.2-dirty installed
build Generate app image from source code global v0.16.0 installed
builder Build Tanzu components global v1.5.0 installed
cluster Kubernetes cluster operations kubernetes v0.31.1 not installed
cluster Plugin for operating clusters in the organization operations v0.11.6 installed
clustergroup A group of Kubernetes clusters operations v0.7.5 installed
domain tanzu-cli domain plugin global v0.1.0 installed
domain-binding tanzu-cli domain binding plugin global v0.1.4 installed
egress Provides crud operations for egresspoints global v0.0.5 installed
ekscluster ekscluster plugin is used to provision and manage the tmc eks clusters. operations v0.6.5 installed
feature Operate on features and featuregates kubernetes v0.31.1 not installed
kubectl-plugin Provides the kubectl functionality global v0.0.0 installed
kubernetes-release Kubernetes release operations kubernetes v0.31.1 not installed
kubetool Invokes any binary after setting KUBECONFIG to match the tanzu CLI global v0.0.0 installed
management-cluster A registered Management cluster operations v0.6.5 installed
package tanzu package management kubernetes v0.35.8 installed
policy Policy management for resources operations v0.6.5 installed
project Create, view, list, use and delete Tanzu projects. global v0.0.0-dev-7741e5e installed
provider-eks-cluster provider-eks-cluster plugin is used to manage and unmanage tmc provider eks operations v0.6.5 installed
clusters
rbac Tanzu Platform Role-Based Access Control global v0.0.0-dev-cde9be5 installed
resource manage resources in a Kubernetes cluster global v0.5.1 installed
secret Tanzu secret management kubernetes v0.33.5 installed
services Commands for working with services, classes and claims kubernetes v0.14.0 installed
space Tanzu space lifecycle management global v0.4.0 installed
telemetry configure cluster-wide settings for vmware tanzu telemetry global v1.1.0 installed
Note: As shown above, some recommended plugins have not been installed or are outdated. To install them please run 'tanzu plugin sync'.
# Confirm the serverPlugin section is properly removed
$ grep -A4 serverPlugin ~/.cache/tanzu/catalog.yaml
$
# Install the context-scope plugins again but using plugin sync with the old CLI
$ tanzu plugin sync
[i] Plugin sync will be performed for context: 'tkg1'
[i] Checking for required plugins for context 'tkg1'...
[i] The following plugins will be installed for context 'tkg1' of contextType 'kubernetes':
NAME TARGET VERSION
cluster kubernetes v0.31.1
feature kubernetes v0.31.1
kubernetes-release kubernetes v0.31.1
[i] Installed plugin 'cluster:v0.31.1' with target 'kubernetes' (from cache)
[i] Installed plugin 'feature:v0.31.1' with target 'kubernetes' (from cache)
[i] Installed plugin 'kubernetes-release:v0.31.1' with target 'kubernetes' (from cache)
[i] Successfully installed all required plugins
[ok] Done
# Now change context so that the tkg context which specifies the
# context-scope plugins is no longer active
$ tanzu context use Tanzu_Platform_for_K8s_SaaS_-_Internal_1
[i] Successfully activated context 'Tanzu_Platform_for_K8s_SaaS_-_Internal_1' of type 'tanzu' (Project: bagreda).
[i] Checking for required plugins for context 'Tanzu_Platform_for_K8s_SaaS_-_Internal_1'...
[i] All required plugins are already installed and up-to-date
# Run the new CLI and notice that the cluster, feature, kubernetes-release plugins
# are properly migrated even though their context was not active.
# This is important because the migration will not be run again.
$ tz plugin list
Some initialization of the CLI is required.
Let's set things up for you. This will just take a few seconds.
Refreshing the 27 installed plugins...
Initialization done!
==
NAME DESCRIPTION TARGET INSTALLED STATUS
apply Apply a resource file operations v0.6.5 installed
apps Applications on Kubernetes kubernetes v0.13.2 installed
appsv2 Applications on Tanzu Platform global v0.21.2-dirty installed
build Generate app image from source code global v0.16.0 installed
builder Build Tanzu components global v1.5.0 installed
cluster Kubernetes cluster operations kubernetes v0.31.1 installed
cluster Plugin for operating clusters in the organization operations v0.11.6 installed
clustergroup A group of Kubernetes clusters operations v0.7.5 installed
domain tanzu-cli domain plugin global v0.1.0 installed
domain-binding tanzu-cli domain binding plugin global v0.1.4 installed
egress Provides crud operations for egresspoints global v0.0.5 installed
ekscluster ekscluster plugin is used to provision and manage the tmc eks clusters. operations v0.6.5 installed
feature Operate on features and featuregates kubernetes v0.31.1 installed
kubectl-plugin Provides the kubectl functionality global v0.0.0 installed
kubernetes-release Kubernetes release operations kubernetes v0.31.1 installed
kubetool Invokes any binary after setting KUBECONFIG to match the tanzu CLI global v0.0.0 installed
management-cluster A registered Management cluster operations v0.6.5 installed
package tanzu package management kubernetes v0.35.8 installed
policy Policy management for resources operations v0.6.5 installed
project Create, view, list, use and delete Tanzu projects. global v0.0.0-dev-7741e5e installed
provider-eks-cluster provider-eks-cluster plugin is used to manage and unmanage tmc provider eks operations v0.6.5 installed
clusters
rbac Tanzu Platform Role-Based Access Control global v0.0.0-dev-cde9be5 installed
resource manage resources in a Kubernetes cluster global v0.5.1 installed
secret Tanzu secret management kubernetes v0.33.5 installed
services Commands for working with services, classes and claims kubernetes v0.14.0 installed
space Tanzu space lifecycle management global v0.4.0 installed
telemetry configure cluster-wide settings for vmware tanzu telemetry global v1.1.0 installed
$ grep -A4 serverPlugin ~/.cache/tanzu/catalog.yaml
Now, confirm that CLI is faster. Before the PR we see a speed normally just above 0.20 seconds.
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.24s user 0.03s system 137% cpu 0.199 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.26s user 0.03s system 136% cpu 0.214 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.25s user 0.03s system 141% cpu 0.195 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.26s user 0.03s system 130% cpu 0.227 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.26s user 0.03s system 140% cpu 0.208 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.24s user 0.02s system 137% cpu 0.189 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.25s user 0.03s system 130% cpu 0.211 total
With the PR we see a time closer to 0.14 which is a 30% improvement
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.12s user 0.02s system 116% cpu 0.123 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.12s user 0.02s system 77% cpu 0.185 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.12s user 0.02s system 115% cpu 0.120 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.12s user 0.02s system 101% cpu 0.141 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.13s user 0.02s system 111% cpu 0.133 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.12s user 0.02s system 108% cpu 0.134 total
$ time tz version>/dev/null
/Users/kmarc/bin/tanzu version > /dev/null 0.12s user 0.02s system 120% cpu 0.121 total
Release note
Speed up CLI by avoiding unnecessary writing to the Catalog of plugins
Additional information
Special notes for your reviewers
On my M1 machine I see these speed results:
I repeatedly run time tz __complete '' > /dev/null to test shell completion responsiveness.
With 1 context: v1.2.0 -> ~0.18 secs v1.3.0 -> ~0.18 secs v1.4.1 -> ~0.16 secs v1.5.1 -> ~0.17 secs main (92c98e8b8) -> ~0.17 secs with this PR -> ~0.10 secs
So we see a significant improvement. It makes basic shell completion blazingly fast.
Thanks for the clear explanation. Looks like a good bit of optimization.
with this PR, since we only run the plugin migration once, we must migrate the context-scoped plugins for all contexts immediately. This is a slight change in behaviour because context-scoped plugins for an inactive (and possibly old context) will now appear as standalone plugins right away for the new CLI. Since this new behaviour is similar to point 2 above, I feel it is acceptable
On point 3, I am not sure whether there was some nuance behind the way it was done before this change. Interested to hear @anujc25's take on this.
On point 3, I am not sure whether there was some nuance behind the way it was done before this change. Interested to hear @anujc25's take on this.
Yes. This seems like the correct thing to do. I added a TODO to update the implementation this way because the globalinitializer approach was not available when I implemented this context-scoped plugin to standalone plugin migration. See the existing TODO below.
// Migrate context-scoped plugins as standalone plugin if required // TODO(anujc): Think on how to invoke this function just once after the newer version // of the CLI gets installed as we just need to do this migration once
