Custom Java -> JS Conversion

[hashtag-smashtag]

The new HostAccess.Builder#targetTypeMapping API (see https://github.com/graalvm/graaljs/issues/94) is fantastic for JS -> Java conversion. Thank you!

Can we similarly handle Java -> JS conversion? I tried using .targetTypeMapping(MyPojo.class, Value.class, ..., ...), but it was not triggered when:

• Using context.getBindings("js").putMember("myPojo", myPojoInstance), but this probably makes sense, because it's added as a host object
• Calling value.execute(myPojoInstance) where value is a JS function accepting an arg.

Does targetTypeMapping only apply to JS -> Java conversions? If it does apply to Java -> JS conversions, how can I correctly use it?

[hashtag-smashtag]

Bump... Any thoughts @chumer ? Is targetTypeMapping only for JS -> Java? Is there any equivalent Java -> JS mechanism? Thanks.

[chumer]

No there is currently no equivalent mapping mechanism from Host Java to JS. Did you try to use a base class for the pojo classes that implement ProxyObject? That should allow you to customize how the guest application sees the object. The best thing about that approach is that it does not require any conversion.

[hashtag-smashtag]

Thanks for the response. Yes we've been using ProxyObject... a lot. But we might be able to eliminate a bunch of proxies with equivalent Java -> JS type mapping.

One example involves an object with a date, where we want it to be both a Java Date and a JS Date. It needs conversion even when the parent object implements ProxyObject. I suppose one might be able to make some complex 'Date' proxy that fulfills both Java and JS without conversion... but that doesn't sound too palatable...

[chumer]

We have plans to add first-class date support in interop. So we will do that conversion for you. (actually we won't convert, we will just interpret Java dates as JS dates and vice versa). You have another use-case?

[mdsina]

@chumer do you have any ETA when Date support will be released?

Currently I have an issue with passing js Date object to java, because graal mapped them to long value. Looks like it cannot be handled though targetMapping

[hashtag-smashtag]

@mdsina For JS-Date to Java-Date, HostAccess.Builder#targetTypeMapping does work for us. We do need to use a proxy, though, so a Date created in Java will act (getTime function) as a JS-Date in Javascript. Also note if you're later converting to something like JSON, it would need to be either a long or a string or nested object since JSON has no date-specific type.

[hashtag-smashtag]

@chumer Another simpler case we've encountered is:

• Java POJO with a field that is an enum
• In JS we'd like the resulting object (from the POJO) to be treated as a string primitive (the Enum#name()). We also are actually working with TypeScript so the definition can be a union of strings (corresponding to the enum values)
• For js->java, we can use targetTypeMapping to get the enum from the js string
• For java->js, we're forced to make the POJO a proxy (or wrap it in one) to call name() on the enum. FWIW I tried making an EnumProxy that would simply return a String, but despite https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/proxy/Proxy.html suggesting one could proxy primitives, I couldn't find a way. ProxyNativeObject#asPointer() was not called and is probably for C/native things only. ProxyPrimitive was removed? Doesn't seem like any Proxy will work for this use case

If we had the equivalent java->js type mapping, we wouldn't need every POJO containing an enum to be/use a proxy.

[hashtag-smashtag]

And while the enum and Date are simple example cases, it's the same story anywhere a proxy/conversion is needed... Either the class is replaced with a proxy (everywhere used), or every parent container must become a proxy to handle the field (at least for Java enum <-> JS String, see above).

If we had Java->Js type mapping, such a proxy could be defined once there, and automatically handled by the the system, including nested situations (field of a parent POJO) etc.

Your thoughts, @chumer ? Many thanks.

[hashtag-smashtag]

Hi @chumer , I'm pinging this again since:

• We're still forced to use a ProxyObject for dates, see https://github.com/graalvm/graaljs/issues/200
• Wanted to get your thoughts on the enum use-case above. A string/primitive proxy could help there. Although Java->JS type mapping would still be awesome :)

[zolstein]

I wanted to chime in with another use-case for this that first-class dates don't cover.

I wanted to pass Jackson's JsonNode values into JS, and tried to define a ProxyObject and ProxyArray around the ObjectNode and ArrayNode respectively. This caused a problem, though.

In Jackson, accessing an element of a JsonNode returns another JsonNode, so my ProxyObject object needs to wrap the result in another proxy before returning it from ProxyObject::get. However, if I wrap a new Proxy instance around it, I have the following issue:

Suppose I have an object with a nested object, represented as a JsonNode.

x = { foo: {bar: 'old' } }

I create a proxy around that value and pass it into the following JS function:

function(x) {
    foo1 = x.foo;
    foo2 = x.foo;
    foo1.bar = 'new';
    return [ Object.is(foo1, foo2), foo2.bar];
}

This returns[ false, 'new' ]. ProxyObject::get returned two different proxy objects around the same underlying JsonNode object. JS interprets these as separate objects because it treats the Proxy object as the object to compare by identity, not the underlying object. In order to solve this, I need to store, along with my Graal Context, a mapping of JsonNodes to ProxyObjects to make sure that I always return the same ProxyObject for the same underlying object. This seems tedious, and I understand (maybe incorrectly) that Graal already does this. (Hence why if I pass the same Proxy object into Context::asValue, JS does understand that they represent the same object.)

If Graal provided a way to define a custom mapping for Java objects when passing them into Graal, I could define that function to wrap a JsonNode in a ProxyObject. Graal would then, I presume, track the JsonNode's identity correctly in the already-existing code-path, making this use-case much simpler.

[hashtag-smashtag]

@zolstein Also check out https://github.com/graalvm/graaljs/issues/89

[chumer]

Since there seems to be more interest in this issue. I've targeted this issue for 21.3. Can't promise it will make it though.

[zikifer]

(Originally posted in Slack:)

The biggest use-case for us is we have a TON of Java methods that return Single<T> (the majority of our Java code is RxJava) that need to be converted to Promises. To accomplish this I have to create a wrapper class for each Java class where each method is basically the same thing,

var result = delegate.performWork(param1, param2);
return jsJava.toPromise(result);

Another use-case for us is we have a few Java methods that return List<T>. These don’t seem to behave quite right in JavaScript without wrapping with ProxyArray.fromList(), and even then you can’t use native JavaScript Array functions (such as .map() or .forEach()). So we need a separate class or an overload method that returns Value to create the ProxyArray and then wrap in a JS fragment that’s basically Array.from(list).

[thammoud]

Hi Christian,

We are migrating away from Rhino to GraalJS. Users of our system can create JS formulas that call into Java code. JS formulas can be tagged as "Wrap Java Primitives", in which case the "double" return value from a Java call is the real Java Object. We use Rhino's WrapFactory to decide how a primitive is presented to the JS code. a JS number or a JavaObject.

package graalvm;
import org.graalvm.polyglot.Context;

public class CallbackFromJS2J2 {    
    public void doIt() {
        HostAccess customMappings = HostAccess.newBuilder(HostAccess.ALL)      
                                    // Christian will fix this. Some configuration is needed to decide what the object to use is 
                                    .build();
        
        Context c = Context.newBuilder("js")
                    .allowHostAccess(customMappings)               
                    .build();
        
        c.getBindings("js").putMember("test", new JavaTest());                
        c.eval("js", "print(`${test.doubleTest(999.1).getClass()}`)"); // getClass() should work if it is a real java object
    }
    
    public static class JavaTest {       
        public double doubleTest(Double variable) {
            return variable; 
        }        
    }
    
    public static void main(String[] args) {
        new CallbackFromJS2J2().doIt();
    }
}`

[fdummert]

Hi, great to see some movement here! Our main reason why we still require the whole proxy infrastructure: JDBC 4.2+ has defined automatic type conversion for OffsetDateTime to the respective DB type, TIMESTAMP WITH TIMEZONE. There is no support for ZonedDateTime. As persistence is an important aspect of our framework, it uses internally OffsetDateTime in arbitrarily complex structures of maps/lists and all execution logic is based upon these. As Polyglot has chosen to only support ZonedDateTime, not OffsetDateTime, we've written a whole infrastructure of ProxyMaps, ProxyArrays, and so forth, to pass OffsetDateTime instances as a JS Date by ourselves, and this extends to argument lists of functions, promises, virtually everything.

[gabrielmattar]

Another use-case for us is we have a few Java methods that return List<T>. These don’t seem to behave quite right in JavaScript without wrapping with ProxyArray.fromList(), and even then you can’t use native JavaScript Array functions (such as .map() or .forEach()). So we need a separate class or an overload method that returns Value to create the ProxyArray and then wrap in a JS fragment that’s basically Array.from(list).

Hi, @zikifer, I think what you need is to set the js.foreign-object-prototype option. https://www.graalvm.org/reference-manual/js/Options/

[zikifer]

Another use-case for us is we have a few Java methods that return List<T>. These don’t seem to behave quite right in JavaScript without wrapping with ProxyArray.fromList(), and even then you can’t use native JavaScript Array functions (such as .map() or .forEach()). So we need a separate class or an overload method that returns Value to create the ProxyArray and then wrap in a JS fragment that’s basically Array.from(list).

Hi, @zikifer, I think what you need is to set the js.foreign-object-prototype option. https://www.graalvm.org/reference-manual/js/Options/

Hi @gabrielmattar, thanks for the suggestion! This feature flag does make list -> array conversion better, and I can do without the additional Array.from(list) JavaScript call. However it still requires Java methods to return Value and have the Java list be wrapped with Value.asValue(ProxyArray.fromList()) in order for it to behave like a real JavaScript array (Java Lists don't have .map() or .shift() and such).

[chumer]

(Java Lists don't have .map() or .shift() and such).

@zikifer That might be something worth filing in another issue. I think at least this should be fixed.

[pmlopes]

(Originally posted in Slack:)

The biggest use-case for us is we have a TON of Java methods that return Single (the majority of our Java code is RxJava) that need to be converted to Promises. To accomplish this I have to create a wrapper class for each Java class where each method is basically the same thing,

var result = delegate.performWork(param1, param2);
return jsJava.toPromise(result);

I had a similar case with vertx, async methods return Future. The way I solved it was to use asm to instrument the Future class to implement the Proxy interface and have a then() method which makes these objects js thennable and so, usable in js promises.