cluster-api-provider-vsphere icon indicating copy to clipboard operation
cluster-api-provider-vsphere copied to clipboard

Published 20 hours ago • verified publisher icon kubernetes-sigs

Document permissions required for CAPV resources

Open srm09 opened this issue 3 years ago • 22 comments
trafficstars

/kind documentation

Describe the solution you'd like It would be good to have a documentation around the permissions required for:

  • user for running controller-manager
  • user for creating clusters.

Anything else you would like to add: n/a

Environment: n/a

srm09 avatar Jan 13 '22 06:01 srm09

/milestone Next

srm09 avatar Jan 13 '22 06:01 srm09

Make sure to add the permissions for the Tagging vAPI.

srm09 avatar Jan 20 '22 18:01 srm09

/good-first-issue /help

srm09 avatar Feb 02 '22 07:02 srm09

@srm09: This request has been marked as suitable for new contributors.

Guidelines

Please ensure that the issue body includes answers to the following questions:

  • Why are we solving this issue?
  • To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
  • Does this issue have zero to low barrier of entry?
  • How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-good-first-issue command.

In response to this:

/good-first-issue /help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Feb 02 '22 07:02 k8s-ci-robot

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar May 03 '22 08:05 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Jun 02 '22 08:06 k8s-triage-robot

/remove-lifecycle rotten

ditsuke avatar Jun 10 '22 15:06 ditsuke

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Sep 08 '22 16:09 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Oct 08 '22 16:10 k8s-triage-robot

/remove-lifecycle rotten /lifecycle frozen

srm09 avatar Oct 27 '22 22:10 srm09

if this issue is still valid. I would like to take this on. Provide me some resources like what would you like to see or expect in this issue. I would love to contribute here.

Sajiyah-Salat avatar Feb 23 '23 10:02 Sajiyah-Salat

@Sajiyah-Salat Thanks for stepping up to move this ahead. Basically what we are looking for here,

  1. Minimum set of permissions required by the vCenter user used to run the CAPV deployment a. Permissions required by the user to create a cluster using the templates/cluster-template.yaml
  2. Any extra set of permissions required for specialized cluster creation, for example, cluster that uses a storage policy instead of a datastore or cluster that leverages the VM tagging functionality.

We are looking to get a doc started with point number 1 and eventually make it cover any extra permissions necessary for specialized use cases..

srm09 avatar Feb 23 '23 22:02 srm09

/remove-lifecycle frozen /lifecycle active /assign @Sajiyah-Salat

srm09 avatar Feb 23 '23 22:02 srm09

FYI:

  • I think the permissions for deploying clusters are specified here: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.6/vmware-tanzu-kubernetes-grid-16/GUID-mgmt-clusters-vsphere.html#vsphere-permissions
  • I think the permissions for the CNS user are specified here: https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-storage/GUID-AEB07597-F303-4FDD-87D9-0FDA4836E5BB.html

These seem to be working for us.

merlijn-sebrechts avatar Mar 13 '23 20:03 merlijn-sebrechts

FYI:

  • I think the permissions for deploying clusters are specified here: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.6/vmware-tanzu-kubernetes-grid-16/GUID-mgmt-clusters-vsphere.html#vsphere-permissions
  • I think the permissions for the CNS user are specified here: https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-storage/GUID-AEB07597-F303-4FDD-87D9-0FDA4836E5BB.html

These seem to be working for us.

Hello @merlijn-sebrechts do you think that documenting permissions required for capv resources is not necessary in the persence of above specified docs?

Sajiyah-Salat avatar Mar 14 '23 01:03 Sajiyah-Salat

Maybe a reference in the capv documentations shall be enough?

PatrickLaabs avatar Mar 14 '23 15:03 PatrickLaabs

  • For permissions for deploying clusters, I think it's best to duplicate the information. The VMware documentation is clearly about a different product (VMware Tanzu), and it's for a slightly different use-case (deploying mgmt cluster, not deploying child clusters).
  • For permissions for the CNS account, a link to the documentation should be enough, in my opinion. The documentation is clear and doesn't use confusing terminology. However, capv should also provide docs on how to use separate accounts for CNS and deploying clusters. Since the CNS credentials are part of the deployed clusters, it's best to give these as little permissions as possible.

merlijn-sebrechts avatar Mar 14 '23 15:03 merlijn-sebrechts

/assign @sbueringer

chrischdi avatar Aug 17 '23 17:08 chrischdi

Useful tool to record privileges https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-security/GUID-8A5066BB-BE0C-48AD-9DDB-F3446AD3B9F6.html

But probably hard to hit all edge cases 😀

sbueringer avatar Sep 14 '23 06:09 sbueringer

Currently testing out a limited privilege account.

I'm using the following docs as reference: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/2.3/tkg-deploy-mc/mgmt-reqs-prep-vsphere.html?hWord=N4IghgNiBcIEoFMCOBXAlgJwQEwAQAUEMBbNAZzLQHsA7M3AMyo1wBcALBXANwGUAHTllwBBAMZiqKGqxABfIA#required-permissions-for-the-vsphere-account-5

However, I did also have to add some Read-only permissions to the hostswitch for now (not finally verified).

Details:

$ govc role.usage | grep less-privileged-user
Role        Entity                                                    Principal                          Propagate
CAPV-ROLE   /dc0                                                      LDAP.LOCAL\less-privileged-user    No
CAPV-ROLE   /dc0/datastore/WorkloadDatastore                          LDAP.LOCAL\less-privileged-user    No
CAPV-ROLE   /dc0/host/cluster01                                       LDAP.LOCAL\less-privileged-user    No
CAPV-ROLE   /dc0/network/some-network                                 LDAP.LOCAL\less-privileged-user    Yes
CAPV-ROLE   /                                                         LDAP.LOCAL\less-privileged-user    No
CAPV-ROLE   /dc0/vm/Templates                                         LDAP.LOCAL\less-privileged-user    Yes
CAPV-ROLE   /dc0/vm/Workloads/some-directory                          LDAP.LOCAL\less-privileged-user    Yes
CAPV-ROLE   /dc0/host/cluster01/10.2.32.7                             LDAP.LOCAL\less-privileged-user    No
CAPV-ROLE   /dc0/host/cluster01/10.2.32.6                             LDAP.LOCAL\less-privileged-user    No
CAPV-ROLE   /dc0/host/cluster01/10.2.32.4                             LDAP.LOCAL\less-privileged-user    No
CAPV-ROLE   /dc0/host/cluster01/10.2.32.5                             LDAP.LOCAL\less-privileged-user    No
CAPV-ROLE   /dc0/host/cluster01/Resources/Compute-ResourcePool/test   LDAP.LOCAL\less-privileged-user    Yes
ReadOnly    /dc0/network/some-hostswitch                              LDAP.LOCAL\less-privileged-user    No

$ govc role.ls CAPV-ROLE
Cns.Searchable
Datastore.AllocateSpace
Datastore.Browse
Datastore.FileManagement
Network.Assign
Resource.AssignVMToPool
Sessions.GlobalMessage
Sessions.ValidateSession
StorageProfile.View
System.Anonymous
System.Read
System.View
VApp.Import
VirtualMachine.Config.AddExistingDisk
VirtualMachine.Config.AddNewDisk
VirtualMachine.Config.AddRemoveDevice
VirtualMachine.Config.AdvancedConfig
VirtualMachine.Config.CPUCount
VirtualMachine.Config.ChangeTracking
VirtualMachine.Config.DiskExtend
VirtualMachine.Config.EditDevice
VirtualMachine.Config.Memory
VirtualMachine.Config.RawDevice
VirtualMachine.Config.RemoveDisk
VirtualMachine.Config.Settings
VirtualMachine.Interact.PowerOff
VirtualMachine.Interact.PowerOn
VirtualMachine.Inventory.CreateFromExisting
VirtualMachine.Inventory.Delete
VirtualMachine.Provisioning.DeployTemplate
VirtualMachine.Provisioning.DiskRandomRead
VirtualMachine.Provisioning.GetVmFiles
VirtualMachine.State.CreateSnapshot
VirtualMachine.State.RemoveSnapshot

chrischdi avatar Sep 18 '23 10:09 chrischdi

Hey everyone, we did also some testing for what are the best least-permissions to run CAPV on our vSphere.

We gave our dedicated user a new permission role, which includes the following:

govc role.ls Kubernetes-API
Cns.Searchable
Datastore.AllocateSpace
Datastore.Browse
Datastore.FileManagement
Global.DisableMethods
Global.EnableMethods
Global.Licenses
Host.Config.Storage
Network.Assign
Resource.AssignVMToPool
Sessions.ValidateSession
StorageProfile.View
System.Anonymous
System.Read
System.View
VApp.Import
VirtualMachine.Config.AddExistingDisk
VirtualMachine.Config.AddNewDisk
VirtualMachine.Config.AddRemoveDevice
VirtualMachine.Config.AdvancedConfig
VirtualMachine.Config.CPUCount
VirtualMachine.Config.DiskExtend
VirtualMachine.Config.Memory
VirtualMachine.Config.RawDevice
VirtualMachine.Config.ReloadFromPath
VirtualMachine.Config.Settings
VirtualMachine.Interact.PowerOff
VirtualMachine.Interact.PowerOn
VirtualMachine.Inventory.CreateFromExisting
VirtualMachine.Inventory.Delete
VirtualMachine.Provisioning.CloneTemplate
VirtualMachine.Provisioning.DeployTemplate
VirtualMachine.State.CreateSnapshot
VirtualMachine.State.RemoveSnapshot

These seems to work perfectly for us.

To mention: We did not include the permissions to build a new image, upload it and create a template out of it. This will be done by a seperate user.

Hope this helps!

Best Patrick

PatrickLaabs avatar Nov 06 '23 07:11 PatrickLaabs

FYI: the role used in CI (which propably has a bit more permissions than required for only provisioning):

❯ govc role.ls capv-ci
Cns.Searchable
Datastore.AllocateSpace
Datastore.Browse
Datastore.FileManagement
Folder.Create
Folder.Delete
Global.SetCustomField
Network.Assign
Resource.AssignVMToPool
Resource.CreatePool
Resource.DeletePool
Sessions.GlobalMessage
Sessions.ValidateSession
StorageProfile.View
System.Anonymous
System.Read
System.View
VApp.ApplicationConfig
VApp.Import
VApp.InstanceConfig
VirtualMachine.Config.AddExistingDisk
VirtualMachine.Config.AddNewDisk
VirtualMachine.Config.AddRemoveDevice
VirtualMachine.Config.AdvancedConfig
VirtualMachine.Config.Annotation
VirtualMachine.Config.CPUCount
VirtualMachine.Config.ChangeTracking
VirtualMachine.Config.DiskExtend
VirtualMachine.Config.EditDevice
VirtualMachine.Config.HostUSBDevice
VirtualMachine.Config.ManagedBy
VirtualMachine.Config.Memory
VirtualMachine.Config.RawDevice
VirtualMachine.Config.RemoveDisk
VirtualMachine.Config.Resource
VirtualMachine.Config.Settings
VirtualMachine.Config.SwapPlacement
VirtualMachine.Config.UpgradeVirtualHardware
VirtualMachine.Interact.ConsoleInteract
VirtualMachine.Interact.DeviceConnection
VirtualMachine.Interact.PowerOff
VirtualMachine.Interact.PowerOn
VirtualMachine.Interact.SetCDMedia
VirtualMachine.Interact.SetFloppyMedia
VirtualMachine.Inventory.Create
VirtualMachine.Inventory.CreateFromExisting
VirtualMachine.Inventory.Delete
VirtualMachine.Provisioning.Clone
VirtualMachine.Provisioning.CloneTemplate
VirtualMachine.Provisioning.CreateTemplateFromVM
VirtualMachine.Provisioning.DeployTemplate
VirtualMachine.Provisioning.DiskRandomRead
VirtualMachine.Provisioning.GetVmFiles
VirtualMachine.State.CreateSnapshot
VirtualMachine.State.RemoveSnapshot

chrischdi avatar May 16 '24 07:05 chrischdi