Getting Forbidden error during secret creation

[vitjok27]

Hello

Running 2.1.2 plugin version and getting 403 while deploying to on-premise k8s cluster. Looks like it is forbidden to create registry secret. See full error output below:

Starting Kubernetes deployment ERROR: ERROR: java.lang.RuntimeException: io.kubernetes.client.ApiException: Forbidden hudson.remoting.ProxyException: java.lang.RuntimeException: io.kubernetes.client.ApiException: Forbidden at com.microsoft.jenkins.kubernetes.wrapper.ResourceManager.handleApiExceptionExceptNotFound(ResourceManager.java:180) at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$SecretUpdater.getCurrentResource(V1ResourceManager.java:776) at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$SecretUpdater.getCurrentResource(V1ResourceManager.java:764) at com.microsoft.jenkins.kubernetes.wrapper.ResourceManager$ResourceUpdater.createOrApply(ResourceManager.java:93) at com.microsoft.jenkins.kubernetes.wrapper.KubernetesClientWrapper.handleResource(KubernetesClientWrapper.java:289) at com.microsoft.jenkins.kubernetes.wrapper.KubernetesClientWrapper.createOrReplaceSecrets(KubernetesClientWrapper.java:336) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.doCall(DeploymentCommand.java:159) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.call(DeploymentCommand.java:124) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.call(DeploymentCommand.java:106) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at hudson.remoting.Engine$1$$Lambda$2/482129899.run(Unknown Source) at java.lang.Thread.run(Thread.java:745) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from <..removed...> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:955) at hudson.FilePath.act(FilePath.java:1162) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand.execute(DeploymentCommand.java:68) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand.execute(DeploymentCommand.java:45) at com.microsoft.jenkins.azurecommons.command.CommandService.runCommand(CommandService.java:88) at com.microsoft.jenkins.azurecommons.command.CommandService.execute(CommandService.java:96) at com.microsoft.jenkins.azurecommons.command.CommandService.executeCommands(CommandService.java:75) at com.microsoft.jenkins.azurecommons.command.BaseCommandContext.executeCommands(BaseCommandContext.java:77) at com.microsoft.jenkins.kubernetes.KubernetesDeploy.perform(KubernetesDeploy.java:42) at com.microsoft.jenkins.azurecommons.command.SimpleBuildStepExecution.run(SimpleBuildStepExecution.java:54) at com.microsoft.jenkins.azurecommons.command.SimpleBuildStepExecution.run(SimpleBuildStepExecution.java:35) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$$Lambda$169/331562520.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more Caused by: hudson.remoting.ProxyException: io.kubernetes.client.ApiException: Forbidden at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:886) at io.kubernetes.client.ApiClient.execute(ApiClient.java:802) at io.kubernetes.client.apis.CoreV1Api.readNamespacedSecretWithHttpInfo(CoreV1Api.java:26343) at io.kubernetes.client.apis.CoreV1Api.readNamespacedSecret(CoreV1Api.java:26325) at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$SecretUpdater.getCurrentResource(V1ResourceManager.java:773) ... 17 more Prepare Docker container registry secrets with name: jenkins-cd-secret Api call failed with code 403, detailed message: { "kind": "Status", "apiVersion": "v1", "metadata": {

}, "status": "Failure", "message": "secrets "jenkins-cd-secret" is forbidden: User "system:anonymous" cannot get resource "secrets" in API group "" in the namespace "my_namespace"", "reason": "Forbidden", "details": { "name": "jenkins-cd-secret", "kind": "secrets" }, "code": 403 }

[kolobok01]

Getting the same problem with deployments: "message": "deployments.apps \"my-app\" is forbidden: User \"system:anonymous\" cannot get resource \"deployments\" in API group \"apps\" in the namespace

[vitjok27]

does the plugin support kubeconfig with oidc authentication ?

[lewisreay]

The issue is that Jenkins is trying to deploy as an anonymous user that doesn't have any permissions.

You could create a new service account and specify that in the context of kubeconfig credential:

Role object: your rules will vary, but this works for creating services and deployments

apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: ns
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
  namespace: ns
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
- apiGroups: [""]
  resources: ["services"]
  verbs: ["create","delete","get","update"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["create","delete","get","update"]  
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
  namespace: ns
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins

[ajoysinhactc]

I am having the same issue .. I do have service user jenkins .. but still not able to execute kubernetesDeploy(configs: "kubedeploy.yaml", kubeconfigId: 'KUBECONFIG') succesfully ..

Always getting the error as below

Loading configuration: /var/lib/jenkins/workspace/jhipprojectmono/kubedeploy.yaml Api call failed with code 403, detailed message: { "kind": "Status", "apiVersion": "v1", "metadata": {

}, "status": "Failure", "message": "deployments.apps "jhipprojectmono" is forbidden: User "system:anonymous" cannot get resource "deployments" in API group "apps" in the namespace "default"", "reason": "Forbidden", "details": { "name": "jhipprojectmono", "group": "apps", "kind": "deployments" }, "code": 403 } ERROR: ERROR: java.lang.RuntimeException: io.kubernetes.client.openapi.ApiException: Forbidden hudson.remoting.ProxyException: java.lang.RuntimeException: io.kubernetes.client.openapi.ApiException: Forbidden at com.microsoft.jenkins.kubernetes.wrapper.ResourceManager.handleApiExceptionExceptNotFound(ResourceManager.

I do have following cubeconfig file as kubeconfigid attached

apiVersion: v1 clusters:

• cluster: certificate-authority-data: LS0tLS1CRUdJTiBDRVJUS .. server: https://XXXX.yl4.us-east-1.eks.amazonaws.com name: arn:aws:eks:us-east-1:YYYY:cluster/finmanagercluster contexts:
• context: cluster: arn:aws:eks:us-east-1:YYYY:cluster/finmanagercluster user: arn:aws:eks:us-east-1:YYYY:cluster/finmanagercluster name: arn:aws:eks:us-east-1:YYYY:cluster/finmanagercluster current-context: arn:aws:eks:us-east-1:YYYY:cluster/finmanagercluster kind: Config preferences: {} users:
• name: arn:aws:eks:us-east-1:YYYY:cluster/finmanagercluster user: exec: apiVersion: client.authentication.k8s.io/v1alpha1 args: - --region - us-east-1 - eks - get-token - --cluster-name - finmanagercluster command: aws

Any issue on configuration or I am missing the plugin configuration

[waterdrops]

I encountered the same ERROR on AWS EKS

k8s version: 1.14.9
EKS version: eks.8

[poizonhack]

Le problème est que Jenkins essaie de se déployer en tant qu'utilisateur anonyme qui n'a aucune autorisation.

Vous pouvez créer un nouveau compte de service et spécifier que dans le contexte des informations d'identification kubeconfig:

Objet de rôle: vos règles varient, mais cela fonctionne pour la création de services et de déploiements

apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: ns
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
  namespace: ns
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
- apiGroups: [""]
  resources: ["services"]
  verbs: ["create","delete","get","update"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["create","delete","get","update"]  
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
  namespace: ns
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins

It works