netbeans icon indicating copy to clipboard operation
netbeans copied to clipboard

Output encoding in Windows uses cp-1252 by default instead of utf-8

Open nerestaren opened this issue 3 years ago • 3 comments

Apache NetBeans version

Apache NetBeans 15

What happened

If you try to output any text in UTF-8, it is not displayed correctly in Windows with JDK19, as it assumes the default of cp1252.

It works with JDK17.

How to reproduce

Just create any project and output any special character that cp1252 and utf-8 encode differently.

// Note that this assumes that your file is saved as utf-8, which is the default.
System.out.println("á");
System.out.println("\uD83D\uDC7D");
System.out.println("🌚");
System.out.println("\u4e16\u754c");

Displays:

�
?
?
??

Did this work correctly in an earlier version?

Apache NetBeans 12.6

Operating System

Windows 10, Windows 11. 64 bit

JDK

JDK19

Apache NetBeans packaging

Apache NetBeans provided installer

Anything else

        System.out.println("Charset.defaultCharset(): " + Charset.defaultCharset());
        System.out.println("file.encoding property:" + System.getProperty("file.encoding"));
        System.out.println("native.encoding property:" + System.getProperty("native.encoding"));
        System.out.println("sun.jnu.encoding property:" + System.getProperty("sun.jnu.encoding"));
        System.out.println("stdout.encoding property:" + System.getProperty("stdout.encoding"));
        System.out.println("sun.stdout.encoding property:" + System.getProperty("sun.stdout.encoding"));

Displays:

Charset.defaultCharset(): UTF-8
file.encoding property:UTF-8
native.encoding property:Cp1252
sun.jnu.encoding property:Cp1252
stdout.encoding property:Cp1252
sun.stdout.encoding property:null

What did not work

  • Adding -J-Dnative.encoding=utf-8 -J-Dstdout.encoding=utf-8 -J-Dsun.stdout.encoding=utf-8 in netbeans_default_options in netbeans.conf.

Workarounds

  • You can set the -Dstdout.encoding=utf-8 VM option in the project.
  • You can reset the System.out to make it use utf-8: System.setOut(new PrintStream(System.out, true, "UTF8"));.

What should happen

  • It should work straight out of the box.
  • Or, at least, there should be a way to make utf-8 the default (setting something in netbeans.conf).

Are you willing to submit a pull request?

No

Code of Conduct

Yes

nerestaren avatar Oct 11 '22 13:10 nerestaren

This is related to #4261 and #4396. See comments about JAVA_TOOL_OPTIONS, etc. too.

It is currently expected that you set the encoding for your project explicitly or via environment. My inclination is that this is the right default behaviour. Perhaps we should consider an option to set for all projects? cc @duoduobingbing

neilcsmith-net avatar Oct 11 '22 14:10 neilcsmith-net

Hmmm... yes, the JAVA_TOOL_OPTIONS can also be used as a workaround:

Picked up JAVA_TOOL_OPTIONS: -Dstdout.encoding=utf-8
á

Anyways, I feel that this should work out of the box.

Thank you!

nerestaren avatar Oct 11 '22 14:10 nerestaren

Anyways, I feel that this should work out of the box.

That has to be balanced with how the JDK works out of the box.

neilcsmith-net avatar Oct 11 '22 14:10 neilcsmith-net