Many tests fail on JDK 11

[mbien]

There are still many tests which can't be run on JDK 11.

NB is currently leaning into the compatibility promise of OpenJDK - it would be good to repair the tests so that the project can say with confidence that things are actually working as intended on JDK 11+.

#4884 is a dummy PR which bumps everything to 11 to generate a current list of failed jobs

feel free to re trigger the workflow as needed when updates happen

remaining list of failing modules:

enterprise/websvc.editor.hints
enterprise/j2ee.dd.webservice

---

fixed in #5138:

java/classfile
java/whitelist

fixed in #5143:

webcommon/javascript2.jsdoc

fixed in #5144:

ide/db.dataview
ide/schema2beans
ide/xml.wsdl.model

fixed in #5658:

php/php.editor

fixed in #6194:

java/java.openjdk.project 

fixed in #6369:

java/debugger.jpda.truffle

fixed in #6617

platform/openide.filesystems
platform/options.api
platform/openide.awt
platform/core.multiview
platform/favorites

fixed in #7201

apisupport/apisupport.ant

fixed in #7415

groovy/groovy.editor

fixed in #7502

platform/lib.uihandler
platform/netbinox
platform/openide.text
platform/openide.util.ui

[ebarboni]

@mbien jenkins linux jenkins build on jdk 11 shows junit failure too at the same place (some are from long time ago) for example, org.netbeans.core.multiview.MultiViewProcessorTest.testFailsWithoutAnIcon

I'm not sure how to make the GA nice to review, very complicated to get the failure place.

[mbien]

the summary page here (scroll down) also shows a list of test failures (it currently can't show errors, only failures): https://github.com/apache/netbeans/actions/runs/3761176522

I believe one way to approach this would be to fix the easiest candidates first, to make the situation better observable and reduce the noise.

[mbien]

java/classfile
java/whitelist

are fixed

[mbien]

webcommon/javascript2.jsdoc and the IDE modules most likely too

[ebresie]

Silly question, what needs to be called or done to trigger the tests failures locally?

[lkishalmi]

I usually just copy the command from the failed job, and execute that locally. Not necessary all failures are reproducible locally, but it worth to give it a try.

[KacerCZ]

Failing test in php/php.editor is o.n.m.php.editor.csl.FoldingTest in method testPHPTags().

This test should fail also under Linux, because actual result does not match expected result. Expected result of test matches actual code folding when I open test file in NetBeans.

What I don't understand is, that running only method testPHPTags() fails, but running all tests in FoldingTest class is OK.

[mbien]

@KacerCZ i saw the linux PHP job green a few times in previous test runs on JDK 11: https://github.com/apache/netbeans/actions/runs/3761176522 https://github.com/apache/netbeans/actions/runs/3623501729

I haven't investigated any further, I only wrote down what I saw back then.

[mbien]

php/php.editor

fixed

[mbien]

java/debugger.jpda.truffle

should be fixed soon. PR is pending. This will move the GraalVM job to JDK 11 baseline and unlock another blocked PR.

[mbien]

PR pending for:

    openide.filesystems
    options.api
    openide.awt
    core.multiview
    platform/favorites

[mbien]

apisupport/apisupport.ant

fixed in https://github.com/apache/netbeans/pull/7201

[mbien]

PR pending for

groovy/groovy.editor

[mbien]

https://github.com/apache/netbeans/pull/7502 pending for

platform/lib.uihandler
platform/netbinox
platform/openide.text
platform/openide.util.ui

[mbien]

closing since CI is now testing on JDK 17+ exclusively. For tests which are disabled via project properties or other means it can be implied that they are probably not working properly - but this doesn't need this issue to track I think.

tests which are currently locked to JDK 17 are tracked via https://github.com/apache/netbeans/issues/7871 and TODOs in main.yml

good job everyone who helped with this!