Public static Methods not found?

[manticore-projects]

Greetings,

after setting up the Gradle Build, I have tried to compile a simple Java Class and it fails promptly:

Execution failed for task ':wasm'.
> java.util.NoSuchElementException
        at com.manticore.jsqlformatter.JSQLFormatter.appendDelete(JSQLFormatter.java:522)

which refers to a private static method using an Enum separation :

  private static void appendDelete(StringBuilder builder, Delete delete, int indent) {
    int i = 0;

    appendKeyWord(builder, outputFormat, "DELETE", "", " ");

    OracleHint oracleHint = delete.getOracleHint();
    if (oracleHint != null) appendHint(builder, outputFormat, oracleHint.toString(), "", " ");

    List<Table> tables = delete.getTables();

    if (tables != null && tables.size() > 0) {
      int j = 0;
      for (Table table : tables) {
        switch (separation) {                    // This is line 522 where JWebAssembly fails
/*--> */ case AFTER:                            // separation is defined on Class Level as: 
            appendObjectName(                    // private static Separation separation = Separation.BEFORE;
                builder,
                outputFormat,
                table.getFullyQualifiedName(),
                "",
                j < tables.size() - 1 ? ", " : " ");
            break;
          case BEFORE:
          default:
            appendObjectName(
                builder, outputFormat, table.getFullyQualifiedName(), j > 0 ? ", " : "", " ");
            break;
        }

        j++;
      }
    }
    ....

What exactly could be the problem here please?

[Horcrux7]

• Do you have more lines of the stacktrace? With the Gradle command line option --stacktrace you get the stacktrace if there is an error.
• With which Java compiler was it compiled?
• Which compiler version do you use? I have rewritten some things yesterday in the BranchManager. Because the Gradle cache it can be tricky to update it.
• I think it is an exception on the BranchManager which calculate the GOTO in the Java byte code back to programming structure which is valid in WASM. A very complex and fragile part of the compiler.

I think the best for reproducing is if you send the *.class file. Then all conditions like Java version, etc can be ignored.

[manticore-projects]

Greetings.

Thank you for prompt response. I have the Gradle Stacktrace output attached.

JVM is Jetbrains 11 (OpenJDK 11). gradle.zip

I have also the JAR file attached, it stems from JSQLFormatter. jsqlformatter-0.1.12.zip

> Task :JSQLFormatter:wasm FAILED
	compiler: jwebassembly-master-SNAPSHOT.jar
Parsing error with org/apache/felix/bundleplugin/AntPlugin.class in file:/home/are/.gradle/caches/modules-2/files-2.1/org.apache.felix/maven-bundle-plugin/5.1.5/ca60c68ce4ba07ca6a230542dd293d65bc7c360b/maven-bundle-plugin-5.1.5.jar
java.io.IOException: Unknown annotation value type pool type: 101
	at de.inetsoftware.classparser.Annotations.readElementValue(Annotations.java:91)
	at de.inetsoftware.classparser.Annotations.read(Annotations.java:51)
	at de.inetsoftware.classparser.ClassFile.getAnnotation(ClassFile.java:209)
	at de.inetsoftware.jwebassembly.module.ModuleGenerator.prepare(ModuleGenerator.java:182)
	at de.inetsoftware.jwebassembly.module.ModuleGenerator.scanLibraries(ModuleGenerator.java:156)
	at de.inetsoftware.jwebassembly.module.ModuleGenerator.<init>(ModuleGenerator.java:116)
	at de.inetsoftware.jwebassembly.JWebAssembly.compile(JWebAssembly.java:359)
	at de.inetsoftware.jwebassembly.JWebAssembly.compileToBinary(JWebAssembly.java:340)
	at de.inetsoftware.jwebassembly.JWebAssembly.compileToBinary(JWebAssembly.java:308)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)

[manticore-projects]

• Which compiler version do you use? I have rewritten some things yesterday in the BranchManager. Because the Gradle cache it can be tricky to update it.

I have deleted the Gradle Cache and run Gradle with --no-build-cache. JDK-8 would fail:

java.lang.UnsupportedClassVersionError: de/inetsoftware/jwebassembly/gradle/WasmPlugin has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

JDK-11 works, but throws:

java.io.IOException: Unknown annotation value type pool type: 101
	at de.inetsoftware.classparser.Annotations.readElementValue(Annotations.java:91)
	at de.inetsoftware.classparser.Annotations.read(Annotations.java:51)
...

[Horcrux7]

This is the Kotlin error from #41. 101 is a small "e" and was fixed in https://github.com/i-net-software/JWebAssembly/commit/ad93c2c10be6ce25006c893138b4953ab512b356

at de.inetsoftware.classparser.Annotations.readElementValue(Annotations.java:91)

The exception is now in line 114. This means you does not use the latest version: https://github.com/i-net-software/JWebAssembly/blob/master/src/de/inetsoftware/classparser/Annotations.java

You can try:

compilerVersion = 'com.github.i-net-software:jwebassembly:master-97f9fb39a7'

This is the ID from the last commit: https://jitpack.io/#i-net-software/jwebassembly/97f9fb39a7

[manticore-projects]

I am sorry, but after messing around I feel like a lot of more work needs to go into deployment first:

1. The correct Gradle dependencies seem to be:

    implementation 'com.github.i-net-software:JWebAssembly:97f9fb39a7'
    implementation 'com.github.i-net-software:jwebassembly-api:master-SNAPSHOT'

Spelling seems to matter and https://jitpack.io/#i-net-software/jwebassembly/97f9fb39a7 explains it wrong.

2. I pulled the source from GIT and deployed it to my Local Maven Repo. Then removed the references to https://jitpack.io.

I have thought this would make the latest GIT commit available, but no chance:

a) Local Repo shows jwebassembly-compiler-0.4.jar and jwebassembly-compiler-0.4.pom

b) while the dependency asks for JWebAssembly or jwebassembly-api --> how does this all work together please?

c) compilerVersion = 'com.github.i-net-software:jwebassembly:master-97f9fb39a7' does not work, but only complains about:

are@archlinux ~/d/s/J/JSQLFormatter (main) [1]> gradle --no-build-cache wasm
> Task :wasm FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':wasm'.
> Could not resolve all files for configuration ':wasmCompiler'.
   > Could not find com.github.i-net-software:jwebassembly:master-97f9fb39a7.
     Searched in the following locations:
       - file:/home/are/.m2/repository/com/github/i-net-software/jwebassembly/master-97f9fb39a7/jwebassembly-master-97f9fb39a7.pom
       - https://oss.sonatype.org/content/groups/public/com/github/i-net-software/jwebassembly/master-97f9fb39a7/jwebassembly-master-97f9fb39a7.pom
       - https://repo.maven.apache.org/maven2/com/github/i-net-software/jwebassembly/master-97f9fb39a7/jwebassembly-master-97f9fb39a7.pom
       - https://jitpack.io/com/github/i-net-software/jwebassembly/master-97f9fb39a7/jwebassembly-master-97f9fb39a7.pom

[manticore-projects]

After a lot of trial'n error, I got access to that particular commit build. Gradle Build file:

dependencies {
    implementation 'com.github.i-net-software:JWebAssembly:97f9fb39a7'
    implementation 'com.github.i-net-software:jwebassembly-api:master-SNAPSHOT'
}

wasm {
    compilerVersion = 'com.github.i-net-software:jwebassembly:97f9fb39a7'
}

Details matter!

Unfortunately, this did not solve or improve anything. I still get exactly the same error message:

Execution failed for task ':wasm'.
> java.util.NoSuchElementException
        at com.manticore.jsqlformatter.JSQLFormatter.appendDelete(JSQLFormatter.java:522)

* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':wasm'.
        at de.inetsoftware.jwebassembly.gradle.WasmCompiler.compile(WasmCompiler.java:213)
        at de.inetsoftware.jwebassembly.gradle.WasmTask$1.execute(WasmTask.java:225)
        at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator.execute(NormalizingCopyActionDecorator.java:61)
        at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator.execute(DuplicateHandlingCopyActionDecorator.java:47)
        at org.gradle.api.internal.file.copy.CopyActionExecuter.execute(CopyActionExecuter.java:47)
        at org.gradle.api.tasks.AbstractCopyTask.copy(AbstractCopyTask.java:149)
        at jdk.internal.reflect.GeneratedMethodAccessor385.invoke(Unknown Source)

[Horcrux7]

• If you use a local maven repository then use compilerVersion = '+'.
• jwebassembly-api is the native API replacment and the DOM API
• jwebassembly-compiler is the compiler which read the byte code form Java, your code and the libraries and generate the wasm output
• jwebassembly-compiler is saved on JWebAssembly project on Github on historian reasons.
• jitpack does not use maven names. It use the Github project name. This result in de.inetsoftware:jwebassembly-compiler:+ --> com.github.i-net-software:jwebassembly-api:master-SNAPSHOT
• if you set a numeric compiler version then the Gradle plugin use de.inetsoftware:jwebassembly-compiler:{number}
• You are right. The compiler version with commit number and master was wrong. My mistake.

Execution failed for task ':wasm'.
> java.util.NoSuchElementException
        at com.manticore.jsqlformatter.JSQLFormatter.appendDelete(JSQLFormatter.java:522)

I need the file com.manticore.jsqlformatter.JSQLFormatter.class that produce the problem for analyzing. Occur this with a release version of the library?

[Horcrux7]

I can reproduce the problem. I will look into.

[manticore-projects]

Thanks a lot!

I would like to offer two things:

1. I will push tomorrow a branch of JSQLFormatter with WebAssembly Output. This might serve as a full fledged test project, which has everything -- except an UI. In my understanding, it would serve well.

2. I would like to support you with Unit Tests for your project.

3. All the information above is valuable and helpful. I would like to offer a Sphinx Website Template, where this can be gathered.

[Horcrux7]

1. I does not understand how this should work if the compiler fail currently.
2. If you can reduce the problem to the max without extra classes then this can be helpful. I think it is a combination of IF, LOOP, SWITCH and ENUM.
3. Documentation is a large problem because things change frequently on beginning. And pull request are not possible for the wiki. But I can save all markdown texte that I receive.

[Horcrux7]

The original problem is fixed but there is a problem with net.sf.jsqlparser.parser.CCJSqlParser.Statement() and net.sf.jsqlparser.parser.CCJSqlParser.JsonFunction(). Where can I find the source code for it? It is not in the library.

[manticore-projects]

Good Morning and compliments of the day.

Thank you again for your help and effort. JSQLFormatter uses JSQLParser, which is a (self containing) Java Library:

dependencies {
    implementation 'com.github.jsqlparser:jsqlparser:4.5-SNAPSHOT'
    implementation 'com.github.i-net-software:JWebAssembly:master-SNAPSHOT'
    implementation 'com.github.i-net-software:jwebassembly-api:master-SNAPSHOT'
}

Will the JWebAssembly Gradle Plugin detect such dependencies automatically or will I need to create a FAT/Ueber JAR containing all the Classes?

[manticore-projects]

The original problem is fixed

I am sad, that I fail to get this stuff running:

1. I deleted all Gradle cache (this should not be necessar though)
2. I pulled from you GIT repo and deployed to Local Maven Repo
3. I tried the '+' Version for the JWebAssembly Compiler as well as the latest Commit ID f393802418

I still get the same error on Row 522 as shown above.

That said, I have created a JWebAssembly branch of my project: https://github.com/manticore-projects/jsqlformatter/tree/JWebAssembly Maybe you would like to clone it and try to assemble/wasm with Gradle by yourself.

[Horcrux7]

JSQLFormatter uses JSQLParser, which is a (self containing) Java Library:

The problem with the JSqlParser project is that I can't find the sources of CCJSqlParser, Only CCJSqlParserUtils. https://github.com/JSQLParser/JSqlParser/tree/master/src/main/java/net/sf/jsqlparser/parser Then it must be go without the sources, which make it a little difficult.

Will the JWebAssembly Gradle Plugin detect such dependencies automatically or will I need to create a FAT/Ueber JAR containing all the Classes?

Yes, but you need only the dependency of the jwebassembly-api. The compiler is loaded from the wasm task self. The compiler version is declared via the property compilerVersion

I still get the same error on Row 522 as shown above.

I have no idea. Can it be that you have set the dependency and not the compilerVersion?

If set the Gradle command line flag --debug and delete the output that the wasm task is not UPTODATE then you can search for wasmCompiler. In the follow lines you can see which compiler jar is used.

[manticore-projects]

JSQLFormatter uses JSQLParser, which is a (self containing) Java Library:

The problem with the JSqlParser project is that I can't find the sources of CCJSqlParser, Only CCJSqlParserUtils. https://github.com/JSQLParser/JSqlParser/tree/master/src/main/java/net/sf/jsqlparser/parser Then it must be go without the sources, which make it a little difficult.

That's because it is a Parser Generator, meaning the Sources are generated only during the build. I can work around that (when my main problem with the JWebAssembly Compiler is solved).

I still get the same error on Row 522 as shown above.

I have no idea. Can it be that you have set the dependency and not the compilerVersion?

I have used master-SNAPSHOT everywhere possible. Maybe you can quickly clone the Branch and try? https://github.com/manticore-projects/jsqlformatter/tree/JWebAssembly

[Horcrux7]

I have analyze the error and found the cause. It could be the cause of all the problems. At least of several problems. The cause is a SWITCH with calculated/conditional switch value like:

switch ((this.jj_ntk == -1) ? jj_ntk_f() : this.jj_ntk) {

The Java bytecode give only the code position where the switch start with a value from stack. Wasm required the code position on which the calculation of the expression starts. The current JUnit tests was to simple.

I need a new idea how I can calculate/detect the right code position and then implements it. This can take some time.

[Horcrux7]

After fixing many bugs with SWICHES, I give up for now. I have no idea how this can be fixed. It looks like the code is generated at Java byte code level. I don't see how I can represent this in Java Syntax or WebAssembly. It seems that there are back jumps from inside a SWITCH in CCJSqlParser.JsonFunction(). A back jump can only be represent with a loop in Java and WebAssembly. But there is no end of the loop. This result in overlapping structures. This is not possible with Java Syntax and WebAssembly.

But the analyze has help to make some things clearer in JWebAssembly. For example I use blocks with input parameter for the SWITCHES.

[manticore-projects]

Greetings.

Thanks a lot for all your effort and I am glad to have provided a solid use case test. On the Switches: I understand that this is a kind of special code, because its auto-generated by Java CC. Although its perfectly valid Java Code and we do have the source. Also, the Java Byte Code can be de-compiled (e.g. in IntelliJ).

So I do wonder: Would it not be easier to transpile Java Source Code instead of Java Byte Code (and to use an additional De-Compiler when only Byte Code was available)?

Java Source Code can be parsed with Java CC, all what was need was to de-parse/transpile it into WebAssembly. At least this is my oversimplified understanding.

[Horcrux7]

The decompilers that I have try because I does not have the source code have not work. There was an output but this output was not matching the byte code.

The byte code has a structure like:

Loopstart
   switchstart
      case x:
         loopend:
switchend

This is not valid in Java or in WebAssembly.

So I do wonder: Would it not be easier to transpile Java Source Code instead of Java Byte Code (and to use an additional De-Compiler when only Byte Code was available)?

The TeaVM use the concept of transpiling.

Java Source Code can be parsed with Java CC, all what was need was to de-parse/transpile it into WebAssembly. At least this is my oversimplified understanding.

I does not have the Java source code of CCJSqlParser. If you have the source code then make it available and I will look into it again. With a source code with which I can reproduce the problem then I can create simple unit test.

[manticore-projects]

Again, thanks a lot. The source code is auto-generated during the build:

1. Step 1: Java CC builds Java Source Code from Grammar
2. Step 2: Java Source Code is compiled into Java Byte Code

I have the Autobuilt Source Code attached, but you can also Git Pull the Branch I created for your tests. Please let me know any additional help/support I can render for you. generated.zip

[Horcrux7]

Thanks for the sources. This will help.

[Horcrux7]

With the library version that I try the compiler error are solved. Please try it.

It ended now with a dependency to native method because

• it use the Java logger API
• the logger try to load a ResourceBundle
• this add a dependency to URL
• which result in a dependency to ZipFile

Of course ZipFile will not work because WebAssembly can't work with files. You can see the parsing order of methods if you set the debug flag.

If you have any suggestion or patch this is welcome.

[manticore-projects]

Good Morning,

awesome work and much appreciated. I will try as soon as possible. On the Logger/ResourceBundle:

1. I think, a lot of Java libraries will call the Java Logger API
2. It will be cumbersome, to remove such calls from the Java Source
3. So why not blacklist such calls during the translation? E.g. by maintaining a List of Java API calls, which will be ignored, maybe based on Package/Class and/or Method Signature of the Method.

Big THANK YOU and Dankeschoen again!

[Horcrux7]

1. Yes.
2. Yes.
3. This is what the JWebAssembly-API do. Take a look into the different replacements classes: https://github.com/i-net-software/JWebAssembly-API/tree/master/src/de/inetsoftware/jwebassembly/api/java The needed work is to find good replacement points.