docker-ssh-agent
docker-ssh-agent copied to clipboard
jenkinsci
Jenkins crash where building project that contain files with accents
Jenkins and plugins versions report
Jenkins crash with the Malformed input or input contains unmappable characters: /home/jenkins/agent/workspace/src/wp/wp-content/uploads/2017/11/._MiniatureAirDuDésertMarocainMiniatureInWueste-624x964.jpg8672861676435183506.tmp when building a project where some files have accents.
I am using the latest jenkins/ssh-agent:jdk11.
What Operating System are you using (both controller, and any agents involved in the problem)?
The docker image in ubuntu 20.04.3 with the latest jenkins/ssh-agent:jdk11.
Reproduction steps
Build a project where some files have accents
Expected Results
it builds
Actual Results
get error Malformed input or input contains unmappable characters: /home/jenkins/agent/workspace/src/wp/wp-content/uploads/2017/11/._MiniatureAirDuDésertMarocainMiniatureInWueste-624x964.jpg8672861676435183506.tmp
Anything else?
No response
I believe that you need to set the locale of the process to a version that supports the character set you are trying to display. I typically do that by passing the environment variable LANG=C.UTF-8.
Hi @MarkEWaite, how can I pass this environment variable ? I tried via the Jenkinsfile:
pipeline {
agent any
environment {
LANG = 'C.UTF-8'
}
stages {...
But it doesn't seem to work, I still have the same error. I use a multibranch pipeline, I also tried with the module https://plugins.jenkins.io/envinject/ but it doesn't seem to work with multibranch pipeline.
I set LANG=C.UTF-8 on the computers that run my agents and assure that the agent has the locale set correctly when the agent starts.
I don't know if declarative Pipeline sets the environment variables before or after the implicit checkout that precedes all other steps of the PIpeline. If it does, then you could use options { skipDefaultCheckout() } to bypass the implicit checkout and then perform the checkout yourself with checkout scm
I still get the issue, I don't understand why because the env var LANG=C.UTF-8 is correctly set
Started by user gautier
Obtained Jenkinsfile from XXXXXXXXXXXXXXXXXXX
[Pipeline] Start of Pipeline
[Pipeline] node
Running on jenkins_agent in /home/jenkins/agent/workspace/TestProject_staging
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ printenv
JENKINS_HOME=/var/jenkins_home
USER=jenkins
CI=true
NODE_LABELS=jenkins_agent
HUDSON_URL=https://jenkins.url
SHLVL=0
MOTD_SHOWN=pam
OLDPWD=/home/jenkins
HOME=/home/jenkins
LOGNAME=jenkins
NODE_NAME=jenkins_agent
_=/opt/java/openjdk/bin/java
STAGE_NAME=Build
EXECUTOR_NUMBER=0
BUILD_DISPLAY_NAME=#24
HUDSON_HOME=/var/jenkins_home
JOB_BASE_NAME=staging
PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
BUILD_ID=24
LANG=C.UTF-8
BUILD_NUMBER=24
SHELL=/bin/bash
JAVA_HOME=/opt/java/openjdk
JENKINS_AGENT_HOME=/home/jenkins
BRANCH_NAME=staging
[Pipeline] checkout
The recommended git tool is: NONE
Wiping out workspace first.
Cloning the remote Git repository
Cloning with configured refspecs honoured and without tags
> JGit fetch # timeout=10
remote: Enumerating objects
remote: Counting objects
remote: Compressing objects
Receiving objects
Resolving deltas
Avoid second fetch
[Pipeline] }
[Pipeline] // withEnv
Updating references
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deploy)
Stage "Deploy" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins_agent
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
at hudson.remoting.Channel.call(Channel.java:1000)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:143)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:129)
at com.sun.proxy.$Proxy105.execute(Unknown Source)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1358)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /home/jenkins/agent/workspace/TestProject_staging/src/wp/wp-content/uploads/2017/11/._MiniatureAirDuDésertMarocainMiniatureInWueste-624x964.jpg17400724680661781114.tmp
at java.base/sun.nio.fs.UnixPath.encode(UnixPath.java:145)
at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
at java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
at java.base/java.io.File.toPath(File.java:2329)
at org.eclipse.jgit.util.FileUtils.isFile(FileUtils.java:802)
at org.eclipse.jgit.util.FileUtils.canExecute(FileUtils.java:831)
at org.eclipse.jgit.util.FS_POSIX.canExecute(FS_POSIX.java:205)
at org.eclipse.jgit.dircache.DirCacheCheckout.checkoutEntry(DirCacheCheckout.java:1506)
at org.eclipse.jgit.dircache.DirCacheCheckout.doCheckout(DirCacheCheckout.java:563)
at org.eclipse.jgit.dircache.DirCacheCheckout.checkout(DirCacheCheckout.java:467)
at org.eclipse.jgit.api.CheckoutCommand.call(CheckoutCommand.java:248)
at org.jenkinsci.plugins.gitclient.JGitAPIImpl.doCheckoutWithResetAndRetry(JGitAPIImpl.java:354)
at org.jenkinsci.plugins.gitclient.JGitAPIImpl.access$100(JGitAPIImpl.java:137)
at org.jenkinsci.plugins.gitclient.JGitAPIImpl$1.execute(JGitAPIImpl.java:313)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:158)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:151)
at hudson.remoting.UserRequest.perform(UserRequest.java:211)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:376)
at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[Bitbucket] Notifying commit build result
[Bitbucket] Build result notified
Finished: FAILURE
My Jenkinsfile is:
pipeline {
agent any
environment {
LANG = 'C.UTF-8'
}
options {
skipDefaultCheckout()
}
stages {
stage('Build') {
steps {
withEnv (['LANG=C.UTF-8']) {
sh 'printenv'
checkout scm
}
}
}
stage('Deploy') {
when {
anyOf {
branch 'master'
branch 'develop'
branch 'staging'
}
}
steps {
script {
if (env.BRANCH_NAME == 'master') {
env.DEPLOY_TO = 'production'
} else if (env.BRANCH_NAME == 'develop' || env.BRANCH_NAME == 'staging') {
env.DEPLOY_TO = 'staging'
}
}
dir ('deploy') {
Do stuff...
}
}
}
}
}
It could be that C.UTF-8 is not an available locale on your operating system. The command locale -a will list the allowed values for the LANG and LC_ALL environment variables.
It could be that LC_ALL is set to a different value (like C) and that overrides the value of LANG.
It could be that the process that is failing does not have the LANG environment variable set because it is not set by the Pipeline until after the failing process has started.
It could be that the file system that you are using does not support UTF-8 characters in file names, even if the operating system supports them.
You could create a tar file with interesting file names in it and then use a test Pipeline to download the tar file and then unpack it. If that succeeds, then the Pipeline is setting the LANG variable early enough that tar works. If that fails, then something is causing the setting of the LANG variable to be ignored or treated as invalid.
Closing since there has been no response since July 2022 and I'm unable to duplicate the issue