fabric8-platform icon indicating copy to clipboard operation
fabric8-platform copied to clipboard

Published 20 hours ago • verified publisher icon fabric8io

Couldn't create Jenkins project dule to file permission

Open yoshioterada opened this issue 7 years ago • 2 comments

Stack trace javax.servlet.ServletException: java.lang.RuntimeException: java.io.IOException: Failed to create a temporary file in /var/jenkins_home/jobs/test-build at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:796) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:233) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135) at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:206) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: java.io.IOException: Failed to create a temporary file in /var/jenkins_home/jobs/test-build

Template Container: jenkins Image: fabric8/jenkins-docker:2.2.297 Ports: 8080/TCP (http) , 50000/TCP (slave) Mount: data → /var/run/docker.sock Mount: jenkins-docker-cfg → /home/jenkins/.docker Mount: jenkins-jobs → /var/jenkins_home/jobs Mount: jenkins-workspace → /var/jenkins_home/workspace Mount: jenkins-token-232mw → /var/run/secrets/kubernetes.io/serviceaccount

Volumes

jenkins-docker-cfg Type: secret (populated by a Secret when the pod is created) Secret name: jenkins-docker-cfg

jenkins-jobs Type: persistent volume claim (reference to a Persistent Volume Claim) Claim name: jenkins-jobs Mode: read-write

jenkins-workspace Type: persistent volume claim (reference to a Persistent Volume Claim) Claim name: jenkins-workspace Mode: read-write

I login to the running Docker image from OpenShift Pod Terminal. Then I confirmed the permission as follows. And jobs, workspace was mounted as root, as a result, it seems that we can't create the new job on jenkins.

$ cd jenkins_home/
~ $ ls -l
total 112
-rw-r--r-- 1 jenkins jenkins 365 Nov 29 11:53 com.dabsquared.gitlabjenkins.GitLabPushTrigger.xml
-rw-r--r-- 1 jenkins jenkins 604 Nov 29 11:53 com.dabsquared.gitlabjenkins.connection.GitLabConnectionConfig.xml
-rw-r--r-- 1 jenkins jenkins 10606 Nov 29 11:53 config.xml
-rw-r--r-- 1 jenkins jenkins 4692 Nov 29 11:53 copy_reference_file.log
-rw-r--r-- 1 jenkins jenkins 159 Nov 29 11:53 hudson.model.UpdateCenter.xml
-rw-r--r-- 1 root root 352 Nov 9 13:39 hudson.plugins.git.GitSCM.xml
-rw-r--r-- 1 root root 377 Nov 9 13:39 hudson.plugins.git.GitTool.xml
-rw-r--r-- 1 jenkins jenkins 222 Nov 29 11:53 'hudson.plugins.openid.OpenIdLoginService$GlobalConfigurationImpl.xml'
-rw-r--r-- 1 root root 344 Nov 9 13:39 hudson.plugins.timestamper.TimestamperConfig.xml
-rw------- 1 jenkins jenkins 1712 Nov 29 11:53 identity.key.enc
drwxr-xr-x 2 jenkins jenkins 4096 Nov 29 11:53 init.groovy.d
drwxr-xr-x 2 root root 6 Nov 29 11:53 jobs
drwxr-xr-x 4 jenkins jenkins 60 Nov 29 11:54 logs
-rw-r--r-- 1 jenkins jenkins 907 Nov 29 11:53 nodeMonitors.xml
drwxr-xr-x 2 jenkins jenkins 6 Nov 29 11:53 nodes
-rw-r--r-- 1 root root 246 Nov 9 13:39 org.jenkinsci.plugins.gitclient.JGitTool.xml
drwxr-xr-x 103 jenkins jenkins 16384 Nov 29 11:53 plugins
-rw-r--r-- 1 root root 9451 Nov 9 13:39 scriptApproval.xml
-rw-r--r-- 1 jenkins jenkins 64 Nov 29 11:53 secret.key
-rw-r--r-- 1 jenkins jenkins 0 Nov 29 11:53 secret.key.not-so-secret
drwx------ 4 jenkins jenkins 4096 Nov 29 11:53 secrets
drwxr-xr-x 2 jenkins jenkins 4096 Nov 29 11:54 updates
-rw-r--r-- 1 jenkins jenkins 604 Nov 29 11:53 com.dabsquared.gitlabjenkins.connection.GitLabConnectionConfig.xml
-rw-r--r-- 1 jenkins jenkins 10606 Nov 29 11:53 config.xml
-rw-r--r-- 1 jenkins jenkins 4692 Nov 29 11:53 copy_reference_file.log
-rw-r--r-- 1 jenkins jenkins 159 Nov 29 11:53 hudson.model.UpdateCenter.xml
-rw-r--r-- 1 root root 352 Nov 9 13:39 hudson.plugins.git.GitSCM.xml
-rw-r--r-- 1 root root 377 Nov 9 13:39 hudson.plugins.git.GitTool.xml
-rw-r--r-- 1 jenkins jenkins 222 Nov 29 11:53 'hudson.plugins.openid.OpenIdLoginService$GlobalConfigurationImpl.xml'
-rw-r--r-- 1 root root 344 Nov 9 13:39 hudson.plugins.timestamper.TimestamperConfig.xml
-rw------- 1 jenkins jenkins 1712 Nov 29 11:53 identity.key.enc
drwxr-xr-x 2 jenkins jenkins 4096 Nov 29 11:53 init.groovy.d
drwxr-xr-x 2 root root 6 Nov 29 11:53 jobs
drwxr-xr-x 4 jenkins jenkins 60 Nov 29 11:54 logs
-rw-r--r-- 1 jenkins jenkins 907 Nov 29 11:53 nodeMonitors.xml
drwxr-xr-x 2 jenkins jenkins 6 Nov 29 11:53 nodes
-rw-r--r-- 1 root root 246 Nov 9 13:39 org.jenkinsci.plugins.gitclient.JGitTool.xml
drwxr-xr-x 103 jenkins jenkins 16384 Nov 29 11:53 plugins
-rw-r--r-- 1 root root 9451 Nov 9 13:39 scriptApproval.xml
-rw-r--r-- 1 jenkins jenkins 64 Nov 29 11:53 secret.key
-rw-r--r-- 1 jenkins jenkins 0 Nov 29 11:53 secret.key.not-so-secret
drwx------ 4 jenkins jenkins 4096 Nov 29 11:53 secrets
drwxr-xr-x 2 jenkins jenkins 4096 Nov 29 11:54 updates
drwxr-xr-x 2 jenkins jenkins 23 Nov 29 11:53 userContent
drwxr-xr-x 10 jenkins jenkins 4096 Nov 29 11:53 war
drwxr-xr-x 2 jenkins jenkins 6 Nov 29 11:53 workflow-libs
drwxr-xr-x 2 root root 6 Nov 28 16:46 workspace
~ $

yoshioterada avatar Nov 30 '16 04:11 yoshioterada

how did you install fabric8? Looks like a permission issue on the Persistent Volume for jenkins

jstrachan avatar Dec 05 '16 14:12 jstrachan

Following is the detail procedure to install the Fabric8 on OpenShift on Azure.

Install OpenShift origin on CentOS(without Red Hat License) on Azure

  1. Create SSH pivate/public keys.

$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/Users/hogehoge/.ssh/id_rsa): openshift-azure-east_rsa
Enter passphrase (empty for no passphrase): [<—— Just Enter] Enter same passphrase again: [<—— Just Enter] Your identification has been saved in openshift-azure-east_rsa. Your public key has been saved in openshift-azure-east_rsa.pub.

  1. Create Azure Resource Group by using azure command

$ azure group create 'OpenShiftRSG-East' 'Japan East'

info: Executing command group create

  • Getting resource group OpenShiftRSG-East
  • Creating resource group OpenShiftRSG-East
    info: Created resource group OpenShiftRSG-East data: Id: /subscriptions/----/resourceGroups/OpenShiftRSG-East data: Name: OpenShiftRSG-East data: Location: japaneast data: Provisioning State: Succeeded data: Tags: null data:
    info: group create command OK
  1. Config KeyVault

$ azure provider register Microsoft.KeyVault

info: Executing command provider register

  • Registering provider Microsoft.KeyVault with subscription ---- info: provider register command OK
  1. Create KeyVault for OpenShift

$ azure keyvault create --vault-name 'OSKeyVault-East' --resource-group 'OpenShiftRSG-East' --location 'Japan East'

info: Executing command keyvault create

  • Checking pre-condition
  • Creating vault OSKeyVault-East
    info: Created vault OSKeyVault-East data: id "/subscriptions/----/resourceGroups/OpenShiftRSG-East/providers/Microsoft.KeyVault/vaults/OSKeyVault-East" data: name "OSKeyVault-East" data: type "Microsoft.KeyVault/vaults" data: location "Japan West" data: properties sku family "A" data: properties sku name "Standard" data: properties tenantId "----" data: properties accessPolicies 0 tenantId "----" data: properties accessPolicies 0 objectId "---- ([email protected])" data: properties accessPolicies 0 permissions keys 0 "get" data: properties accessPolicies 0 permissions keys 1 "create" data: properties accessPolicies 0 permissions keys 2 "delete" data: properties accessPolicies 0 permissions keys 3 "list" data: properties accessPolicies 0 permissions keys 4 "update" data: properties accessPolicies 0 permissions keys 5 "import" data: properties accessPolicies 0 permissions keys 6 "backup" data: properties accessPolicies 0 permissions keys 7 "restore" data: properties accessPolicies 0 permissions secrets 0 "all" data: properties accessPolicies 0 permissions certificates 0 "all" data: properties enabledForDeployment false data: properties vaultUri "https://OSKeyVault-East.vault.azure.net" warn: This vault does not support HSM-protected keys. Please refer to http://go.microsoft.com/fwlink/?linkid=512521 for the vault service tiers. warn: When creating a vault, specify the --sku parameter to select a service tier that supports HSM-protected keys. info: keyvault create command OK
  1. Set the secret for created KeyVault

$ azure keyvault secret set -u 'OSKeyVault-East' -s 'MySecret' --file ~/.ssh/openshift-azure-east_rsa info: Executing command keyvault secret set

  • Creating secret https://OSKeyVault-East.vault.azure.net/secrets/MySecret
    data: value "-----BEGIN RSA PRIVATE KEY-----

**********************== -----END RSA PRIVATE KEY----- " data: id "https://oskeyvault-east.vault.azure.net/secrets/MySecret/" data: attributes enabled true data: attributes created "2016-12-01T09:55:36.000Z" data: attributes updated "2016-12-01T09:55:36.000Z" info: keyvault secret set command OK

  1. Configure the policy for Key Vault to be able to install via the template

$ azure keyvault set-policy -u 'OSKeyVault-East' --enabled-for-template-deployment true info: Executing command keyvault set-policy

  • Loading vault OSKeyVault-East
  • Updating vault OSKeyVault-East
    info: Vault OSKeyVault-East was updated data: id "/subscriptions/----/resourceGroups/OpenShiftRSG-East/providers/Microsoft.KeyVault/vaults/OSKeyVault-East" data: name "OSKeyVault-East" data: type "Microsoft.KeyVault/vaults" data: location "Japan West" data: properties sku family "A" data: properties sku name "Standard" data: properties tenantId "----" data: properties accessPolicies 0 tenantId "----" data: properties accessPolicies 0 objectId "---- ([email protected])" data: properties accessPolicies 0 permissions keys 0 "get" data: properties accessPolicies 0 permissions keys 1 "create" data: properties accessPolicies 0 permissions keys 2 "delete" data: properties accessPolicies 0 permissions keys 3 "list" data: properties accessPolicies 0 permissions keys 4 "update" data: properties accessPolicies 0 permissions keys 5 "import" data: properties accessPolicies 0 permissions keys 6 "backup" data: properties accessPolicies 0 permissions keys 7 "restore" data: properties accessPolicies 0 permissions secrets 0 "all" data: properties accessPolicies 0 permissions certificates 0 "all" data: properties enabledForDeployment false data: properties enabledForTemplateDeployment true data: properties vaultUri "https://oskeyvault-east.vault.azure.net/" info: keyvault set-policy command OK
  1. Install the OpenShift from the ARM Template (Push “Deploy” button from the link)

https://github.com/Azure/azure-quickstart-templates/tree/master/openshift-origin-rhel

Input parameters :

BASIC: —————————————————————————— Subscription：Microsoft Azure ********* Resource Group：OpenShiftRSG-East (<— The above created existing resource group) Location：Japan East ――――――――――――――――――――――――――

Configuration: ―――――――――――――――――――――――――― _artifacts Location：https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/openshift-origin-rhel/ Master Vm Size：Standard_DS2_v2 Node Vm Size : Standard_DS2_v2 Os Image：centos Openshift Master Hostname：osmaster Openshift Master Public Ip Dns Label：masterdom Node Prefix：nods Node Instance Count：2 Admin Username：hogehoge Admin Password : password

Ssh Public Key：ssh-rsa /***//*********************************************************************************************************// [email protected] (Please get the above public key from the following command $ cat -v ~/.ssh/openshift-azure-east_rsa.pub)

Subscription Id：---- Key Vault Resource Group：OpenShiftRSG-East Key Vault Name：OSKeyVault-East Key Vault Secret：MySecret ――――――――――――――――――――――――――

In order to Login the created system.

$ ssh -i openshift-azure-east_rsa [email protected] The authenticity of host 'osmaster.japaneast.cloudapp.azure.com (52...85)' can't be established. ECDSA key fingerprint is SHA256:*************************************. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'osmaster.japaneast.cloudapp.azure.com,52...85' (ECDSA) to the list of known hosts.

  • If you can’t login with following messages(I faced the following problem), Please refer to the following “Permission denied (publickey,gssapi-keyex,gssapi-with-mic).”

$ azure config mode arm $ azure vm reset-access -g OpenShiftRSG-East -n osmaster -r $ azure vm reset-access -g OpenShiftRSG-East -n osmaster -u hogehoge -p password

※ If you faced some trouble during the installation, please see the following log file on master machine ?

$ sudo ls -l /var/lib/waagent/custom-script/download/0/ Total 204 -r-x------. 1 root root 1655 12月 1 10:25 masterPrep.sh -rw-------. 1 root root 5907 12月 1 10:32 stderr -rw-------. 1 root root 194740 12月 1 10:32 stdout $ sudo ls -l /var/lib/waagent/custom-script/download/1/ Total 12 -r-x------. 1 root root 3387 12月 1 10:34 deployOpenShift.sh -rw-------. 1 root root 68 12月 1 10:34 stderr -rw-------. 1 root root 226 12月 1 10:34 stdout

Please add following environment value

vi ~/.bash_profile

export KUBERNETES_MASTER=https://masterdom.japaneast.cloudapp.azure.com:8443 export KUBERNETES_DOMAIN=52...17.xip.io export KUBERNETES_NAMESPACE=devops

In order to install Fabric8

$ FABRIC8_OS=linux $ FABRIC8_VERSION=0.4.64 $ wget -O gofabric8 https://github.com/fabric8io/gofabric8/releases/download/v$FABRIC8_VERSION/gofabric8-$FABRIC8_OS-amd64 $ chmod +x gofabric8 $ ./gofabric8 version $ ./gofabric8 -s https://masterdom.japaneast.cloudapp.azure.com:8443 --domain=52...17.xip.io -y --namespace="fabric8" deploy $ ./gofabric8 deploy --domain=52...17.xip.io

Default GOGS admin username/password = gogsadmin/RedHat$1

Checking if PersistentVolumeClaims bind to a PersistentVolume .........There are pending PersistentVolumeClaims If using a local cluster run gofabric8 volumes to create missing HostPath volumes

If you see the above message, please execute following? $ ./gofabric8 volumes

If you can’t login to the Fabric8 console due to the OAuth problem, please execute following? $ oc get oauthclient fabric8 -o=yaml > fabric8-oauthclient.yaml

$ vi fabric8-oauthclient.yaml

  • http://fabric8.test.52...17.xip.io
  • https://fabric8.test.52...17.xip.io $ oc apply --validate=false -f fabric8-oauthclient.yaml

$ sudo vi /etc/origin/master/master-config.yaml

corsAllowedOrigins:

  • .* <—— add this code.
  • 127.0.0.1
  • localhost
  • 10.0.0.4
  • osmaster.xnpsfaybbsqedcn0zaijip12vf.lx.internal.cloudapp.net
  • kubernetes.default

$ sudo systemctl restart origin-master.service

yoshioterada avatar Dec 07 '16 05:12 yoshioterada