[Fernflower] New Testing Type: stdout comparison tests (including new test cases)

[droid666]

(resubmitted, had a git accident..., contributor license agreement on the way...)

For these test you just dump java files into the test folder. They will be compiled, then run (output stored), then the classes are decompiled, compiled and run too (output compared).

So much less maintaining to do then with the other tests (for the others all tests have to be updated on fundamental changes to Fernflower that change decompiled output).

These tests don't require that the decompiler output never changes. They check that the output of the classes when run does not change and is the same as for the original classes. Also we get a failure when the decompiled classes cannot be compiled again.

It also supports optional pattern matching on the created classes. Note: the compiler runs with 'renaming' turned on to test that at the same time.

Classes in 'javadefault' are compiled with Java 1.8, in 'java14' with Java 1.4, in 'java15' with Java 1.5 and so on.

At the moment all but the example test are deactivate (so skipped), see build.xml begin of test-stdout-prepare:

<!-- here you can skip tests -->
<patternset id="stdout-tests.exclude.pattern">
  <include name="**/*.java"/>

  <!-- general problems -->
  <!-- non-compilable: causes private field/method access -->
  <exclude name="**/PrivateAccess.java"/>
  <!-- non-compilable: assertion code atm can only handle the throw in the if clause, not in the else clause -->
  <exclude name="**/assertions/**/*"/>
  <!-- overloading causes many troubles... -->
  <exclude name="**/overloading/**/*"/>
  <!-- non-compilable: renaming fails -->
  <exclude name="**/renaming/**/*"/>

  <!-- old java versions -->
  <!-- non-compilable: (.class field trouble with assert) (remove filter on assertions above too) -->
  <exclude name="**/java14/assertions/**/*"/>

</patternset>

[droid666]

Note: I think it needs ant 1.9.6, at least I cannot run it on command line with ant 1.9.4.

[droid666]

I'm thinking about improving this. But only if you want to merge it.

Have a optional file next to each test with additional decompiler arguments (so that only the renaming tests have renaming turned on, and only the overloading tests use the new -osc=1 (if you merge that)).

Also at the moment the test cases are duplicated if they should be compiled with multiple java versions (e.g. in package 'java14' and 'java15'). I could change that. Question is if default is to compile with all java versions? Or the opposite: by default compile with current java (1.8 atm) only? In both cases exceptions would be defined in build.xml.

Also I could maybe remove the optional pattern matching on the decompiled java files. Not sure if this will ever be needed.

[trespasserw]

Hi!

Thank you for the contribution, such tests are indeed useful! However, the decompiler now is a part of IDEA codebase, and I'm hesitant in having such tests in it. It would be better to create a separate repository for them. What do you think?