Unmappable characters in filename break project detection

[rhuitl]

I'm using a Github Codespace with the default image. It has a UTF-8 locale:

$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

In the project there is a file that is using a UTF-8 character in the filename. Until I remove it, the plugin is unable to detect the project. The log file contains the message

java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /home/codespace/my-project/target/test-classes/data-files/Grundfl��chen.dat

To me this looks like the filename is UTF-8 (the character is an ä) but the scanner does not treat it as UTF-8 but rather ASCII.

[jdneo]

Could you share the full stacktrace?

[rhuitl]

Here's a repository that reproduces the issue:

• Open https://github.com/rhuitl/vscode-java-bug as a codespace
• Install the Java support when prompted
• Open the "Java Projects" panel and click "Import Projects"
• Observe the error message
• Click on "Show errors". The panel will tell you "No problems have been detected in the workspace." => maybe there is another bug or missing feature that does not report the problem properly?
• Open Java language server logfile:

!ENTRY org.eclipse.jdt.ls.core 4 0 2023-02-03 10:56:08.877
!MESSAGE Initialization failed 
!STACK 0
java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /workspaces/vscode-java-bug/src/main/resources/file-with-utf8-������.txt
	at java.base/sun.nio.fs.UnixPath.encode(Unknown Source)
	at java.base/sun.nio.fs.UnixPath.<init>(Unknown Source)
	at java.base/sun.nio.fs.UnixFileSystem.getPath(Unknown Source)
	at java.base/java.nio.file.Path.of(Unknown Source)
	at java.base/java.nio.file.Paths.get(Unknown Source)
	at org.eclipse.core.internal.filesystem.local.nio.PosixHandler.fetchFileInfo(PosixHandler.java:41)
	at org.eclipse.core.internal.filesystem.local.LocalFileNativesManager.fetchFileInfo(LocalFileNativesManager.java:65)
	at org.eclipse.core.internal.filesystem.local.LocalFile.fetchInfo(LocalFile.java:161)
	at org.eclipse.core.filesystem.provider.FileStore.fetchInfo(FileStore.java:260)
	at org.eclipse.core.filesystem.provider.FileStore.childInfos(FileStore.java:99)
	at org.eclipse.core.internal.localstore.UnifiedTree.getLocalList(UnifiedTree.java:360)
	at org.eclipse.core.internal.localstore.UnifiedTree.addChildren(UnifiedTree.java:148)
	at org.eclipse.core.internal.localstore.UnifiedTree.addNodeChildrenToQueue(UnifiedTree.java:254)
	at org.eclipse.core.internal.localstore.UnifiedTree.accept(UnifiedTree.java:120)
	at org.eclipse.core.internal.localstore.FileSystemResourceManager.refreshResource(FileSystemResourceManager.java:978)
	at org.eclipse.core.internal.localstore.FileSystemResourceManager.refresh(FileSystemResourceManager.java:961)
	at org.eclipse.core.internal.resources.Resource.refreshLocal(Resource.java:1573)
	at org.eclipse.core.internal.resources.Project.open(Project.java:1094)
	at org.eclipse.core.internal.resources.Project.open(Project.java:1150)
	at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager.create(ProjectConfigurationManager.java:835)
	at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager.lambda$1(ProjectConfigurationManager.java:153)
	at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:371)
	at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:275)
	at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:214)
	at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager.importProjects(ProjectConfigurationManager.java:139)
	at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager.importProjects(ProjectConfigurationManager.java:130)
	at org.eclipse.jdt.ls.core.internal.managers.MavenProjectImporter.importToWorkspace(MavenProjectImporter.java:230)
	at org.eclipse.jdt.ls.core.internal.managers.ProjectsManager.importProjects(ProjectsManager.java:148)
	at org.eclipse.jdt.ls.core.internal.managers.ProjectsManager.initializeProjects(ProjectsManager.java:111)
	at org.eclipse.jdt.ls.core.internal.handlers.InitHandler$1.runInWorkspace(InitHandler.java:242)
	at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:43)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

[jdneo]

Hmm, I tried this issue on Windows but not reproducing it. It's reproducible in codespace. I guess that's related with the Linux encoding issue, but didn't find a workaround so far.

[rhuitl]

I enabled UTF-8 for the devcontainers build image (https://github.com/rhuitl/vscode-java-bug/commit/196bbdeb3f07ac2bbc7e19a468eeb6c9525a4196) and when you start a codespace with that image, the project can be imported: https://github.com/rhuitl/vscode-java-bug/tree/fix

For reference, using the current upstream devcontainer image to start a codespace, you will not be able to import the project: https://github.com/rhuitl/vscode-java-bug/tree/main

So I think this issue is best fixed upstream. You might still want to consider two improvements for this plugin:

• Do not fail the whole project import if individual files have issues
• Report errors during import into the errors list of VSCode