[feature-request] Ability to process Promises from Polyglot applications

[mdsina]

We have project based on Project Reactor (https://projectreactor.io) that have own event-loop for message processing. It's would be great if Polyglot will have an ability to manually process Promises from Polyglot-based applications on HotSpot, as for example. Currently only Node runtime can use internal API for Promises processing.

[eleinadani]

Hi @mdsina,

do you have an example of how such an API would look like?

[mdsina]

Hi @eleinadani I apologize this would like iterable chain of callbacks, that I can rebuild with API of Reactor (or something else) Also there is a case when callback return new callback and we should add them to stack of callbacks chain to process them. Not sure how does Node do that, but looks like if I can grab callbacks from promise we can build own processing on them

[eleinadani]

@mdsina could you maybe provide us a concrete example of JavaScript+Java code for this scenario?

[pmlopes]

I believe what @mdsina wants (and I'd to know if it's possible too) is access the executor as described in https://promisesaplus.com/ sections 2.2.4

Where it's suggested that the platform code should be used.

In vert.x I've added a full promise polyfill to control the executor, but if there's a way to define one for the native implementation then it would simplify everyone's code I guess.

[mdsina]

@pmlopes thanks for info. Looks like Promise resolving is not so easy if I need to follow https://promisesaplus.com/ Also polyfill is a good idea, but in our case we use async\await syntax. Transpile async\await to Promises requires a more complex solution with JS parsing.

@eleinadani A more complex example I can provide later. But I need to access promise value. Currently Promise value is not accessible through public methods.

Context context = ...
Value val = context.eval("js", "async callFunction(ctx) {" +
"await res = util.post(ctx.get('I'))" +
"return res;" +
"}" + 
"return callFunction();"
);

// there are code that get promise value and bind Polyglot Context to reactive stream and execute promise values on context as different tasks on workers

[pmlopes]

async/await can be handled by graaljs the polyfill works for me with async await too.

[mdsina]

@pmlopes Can you please share your polyfill or some part if possible?

[pmlopes]

https://github.com/reactiverse/es4x/blob/develop/es4x/src/main/resources/io/reactiverse/es4x/polyfill/promise.js

[mdsina]

@pmlopes Oh, I see. You just replace defualt with your own implementation. Thanks.

[eleinadani]

Graal.js already supports some interoperability between Java code and JS promises. For example, JS code can await on a Java "thenable" object, and Java code can register a Java lambda function to be called once a JS Promise resolves.

To clarify what is currently supported, we added a few unit tests here

@mdsina from your code above it seems to me that you want to await on a Java object, and return a JS Promise back to Java. As you can see from our unit tests, we might already support this scenario. If not, could you please give us a more concrete exaple of what is missing?

@pmlopes, maybe what you need is also already covered in our tests above? If not, happy to discuss how we could improve our interop support for promises

[pmlopes]

I think what isn't totally clear is where is the executor happening in graaljs (not node). I think some documentation should be added to explain how this works:

var promise = new Promise(function(resolve, reject) {
  resolve("Stuff worked!");
  // so the question is... who runs the line above?
  // can we intercept it to be sure it runs where we would like it to run?
  // say our own executor thread?
});

promise.then(res => print(res));
print("after then, before resolve");

Which outputs the right stuff:

after then, before resolve
Stuff worked!

[pmlopes]

so for my case, I think I can drop the polyfill as the thenable is working, the only concern I still have is since I've many context objects I might need to schedule the execution on the right one in case there's some mismatch.

[eleinadani]

Sure, we can add some documentation. In in general, we execute promise reactions (i.e., resolve("Stuff worked!") in your example) when the JS stack is empty (as prescribed in the spec).

Regarding execution scheduling and context mismatch: yes, you need to make sure that the same JS context is never used by two or more threads concurrently. This does not prevent entering/leaving a context from different threads, though: as long as access is not concurrent, you can use the same context from multiple threads with proper synchronization.

Happy to hear that the 'thenable' solution works for you!

[mdsina]

That looks will not work for me. because I want deffer evaluation to initial subscriber.

In my casse I have reactive stream, that will emit value only on subscription. So in case in handle of then callback I need to put that callback to stream chain.

@pmlopes Polyfill not work in my case too, because graaljs wrap async\await functions with own implementation instead of mine and put DynamicObject<JSPromise> as result

[pmlopes]

I don't know what your use case is. But the type should not be a problem as long the then method is present. But i might not understand the problem here...

[mdsina]

@pmlopes Sorry for the inconvenience. I mean:

js:
async function callFunction() {
      const a =  await ..
}

then in Java:

context.eval(promisePolyfillSource);
context.eval(jsSource); // put callFunction to global as in previous snippet
Value result = context.getBindings("js").getMember("callFunction").execute();

The result will be holder of DynamicObject<JSPromise> When I trying something like that:

context.eval("js", "Promise.resolve(1)"); // returns polyfilled promise

That returns what I need. But not in case with async\await

[pmlopes]

Ok but if you call the then method on the result object with a Java function you can then run your code in the Java thread/stream you want right?

[mdsina]

@pmlopes Yeah, looks working, but why I do not test it first, because it will be more complicated if I want to schedule my task on same scheduler and scheduler was changed to all sequence Currently solution with then cannot handle this, that's why I asking how to do promise handling manually. But looks like that will work in common situations.

At this lines I just return Thenable that executed later: https://github.com/mdsina/graaljs-executor-web-service/commit/ec3ef3a1a30b8559047ba8a89393256953b94e88#diff-6756cb3fe8e4eb4fa80dfe8fe6c56a6eR32

And after that in JS code that looks like this: https://github.com/mdsina/graaljs-executor-web-service/commit/ec3ef3a1a30b8559047ba8a89393256953b94e88#diff-88c8986a4fb5da05073fbdbebe437909R3

You can dig out in other changes in this commit if interested.

[fredho]

Hi @mdsina @eleinadani, sorry to revisit an old issue. Regarding DynamicObject<JSPromise> being held in result, is there a way to extract the JSPromise result from the org.graalvm.polyglot.Value in Java?

My use case is that in my Value, I have a resolved promise in a with the correct PromiseValue when I peer into it with an IDE watch/expression, but I'm unable to extract the PromiseValue with any of the as..() methods on Value. Specifically, when I try to extract the value as com.oracle.truffle.js.runtime.builtin.JSPromise, I get this exception:

superclass access check failed: class com.oracle.truffle.js.runtime.builtins.JSClass (in unnamed module @0x2049ac02) cannot access class com.oracle.truffle.api.object.ObjectType (in module org.graalvm.truffle) because module org.graalvm.truffle does not export com.oracle.truffle.api.object to unnamed module @0x2049ac02

Any tips on what I may be doing incorrectly here? Thank you. I'm using graalvm-ce-java11-20.1.0 as my JRE.

[mdsina]

@fredho Hi. I do not extract Promise from Value, just using it as is:

                    Value promise = context.getBindings("js")
                        .getMember("main")
                        .execute(parsedInputs);

                    Function<Object, Object> errorHandler = createErrorHandler(sink);

                    promise
                        .getMember("then").execute((Function<Object, Object>) o -> {
                            sink.success(o == null ? Collections.emptyMap() : o);
                            return null;
                        }, errorHandler)
                        .getMember("catch").execute(errorHandler)
                        .getMember("then").executeVoid((Function<Object, Object>) o -> {
                            //TODO
                            log.trace("Context closing.");
                            context.close();
                            return null;
                        });

May be mapping to Thenable will work, idk.

---

Maybe @eleinadani have some advice. In code above I close Context on when main Promise is complete. But currently If I still have background Promises, that will not work. It seems to be hard to control over all Promises, because there are may be Promises chains in user code, that I cannot add to my queue or something like that. It would be great if I can access Promises queue.

[eleinadani]

Hi @fredho, we do not provide mechanisms to extract the "internal" value of a resolved promise. You can however register another callback (via then), which will be called with the promise value as suggested above by @mdsina. You can find other examples here

@mdsina I don't think that we can expose the promise queue to users, because that might violate the ECMA spec. I agree that there should be a way to close a context in a "safe" way (i.e., after all promises have been processed): this might require some changes to the Context internals, so it might take a while before we can provide an API for that. I will discuss with the team.

[frank-dspeed]

@eleinadani maybe you can simply consider implementing TLA Top Level Await that will also solve that as this is needed anyway and does unwrap promises.

[eleinadani]

@frank-dspeed we already support top-level await when the used ECMA version is >= ECMAScript2021

[AndreiYu]

Hi @fredho, we do not provide mechanisms to extract the "internal" value of a resolved promise. You can however register another callback (via then), which will be called with the promise value as suggested above by @mdsina. You can find other examples here

@mdsina I don't think that we can expose the promise queue to users, because that might violate the ECMA spec. I agree that there should be a way to close a context in a "safe" way (i.e., after all promises have been processed): this might require some changes to the Context internals, so it might take a while before we can provide an API for that. I will discuss with the team.

Sorry, if the question is duplicate, but is there any possibility to map resolved or rejected JS Promise to Java object using Value.as(MyJavaPojoJs.class)? In our code Value.toString returns something like this: Promise{[[PromiseStatus]]: "rejected", [[PromiseValue]]: ReferenceError: abc is not defined} but I'm unable to map this JS object to Java in order to extract PromiseValue and ReferenceError values

[eleinadani]

@AndreiYu, in your case, couldn't you register a Java reaction method for your promise? In that way, the Java method will be called when a promise is rejected (or resolved successfully), and there would be no need to call value.as(Something.class) to extract its value. We have a few examples here.

[ahmadizm]

I was looking at the examples here, and am running into the following issues when completing promises using two options:

1. The first option appears to be following those examples and allowing complete host access .allowHostAccess(HostAccess.ALL) -- this is not recommended, and should not be done. I tried defining public methods to handle then and catch with HostAccess.EXPLICIT to no avail. Here is a simple example of how I am doing it:

                final Value executionResult = bindings.getMember(methodName)
                         .execute(param);
                executionResult.invokeMember("then", getObjectConsumer(stringWriter));

with:

    @HostAccess.Export
    public Consumer<Object> getObjectConsumer(StringWriter stringWriter) {
        return (value) -> appendEvaluationResult(stringWriter, Value.asValue(value));
    }

Note that this works if I allow all-access like the examples.

2. The other way would be letting the engine complete the promise using:

            .allowExperimentalOptions(true)
            .option(JSContextOptions.INTEROP_COMPLETE_PROMISES_NAME, "true")

which is an experimental feature and is not supposed to be used in production.

So am I correct in assuming that there is no recommended way to handle js promises in production using GraalVM? Thanks for your help.

[iamstolis]

@ahmadizm Your first approach does not work because you are using @HostAccess.Export incorrectly. This annotation should be used on members (fields/methods) that are invoked from your JavaScript code. The annotated method getObjectConsumer() is invoked by your Java code. You should annotate accept() method of your Consumer. Unfortunately, this is not possible when using Java lambdas. On the other hand, I believe that this approach will work if you replace the lambda with a regular class (implementing Consumer) with @HostAccess.Export annotation moved to its accept() method.

[ahmadizm]

Thank you @iamstolis, that indeed works. Just for completion, here is how I changed it:

mappingResult.invokeMember("then",
                            new JSPromiseThenConsumer(stringWriter, objectMapper, log));

with:

public class JSPromiseThenConsumer implements Consumer<Object> {
    public JSPromiseThenConsumer() {
        // constructor 
    }

    @HostAccess.Export
    @Override
    public void accept(Object value){
          // logic to decide what to do with value
    }
}

[AndreiYu]

Is there any way of how to get full async JS stackTrace as we can do in Google Chrome V8 console? For ex. this function

const throwSomething = async () => {
  await async function() {throw new Error()}();
 };
 const fn1 = async () => {
  return await throwSomething();
 };
 fn1();
123

returns

VM254:2 Uncaught (in promise) Error
    at <anonymous>:2:33
    at throwSomething (<anonymous>:2:45)
    at fn1 (<anonymous>:6:16)
    at <anonymous>:8:2

in Chrome console,

but context.eval() in Graal returns either success 123 (ecmascript 2020) latest successful statement without any Promise result or Promise{[[PromiseStatus]]: "resolved", [[PromiseValue]]: undefined} (for ecmascript 2021) - it seems to be 123 just wrapped to Promise, no any Uncaught in promise since Graal doesn't wait throwSomething and fn() to complete as Chrome V8 does.

So how can we handle those uncaught Promises in Graal and see them in async stackTrace?

[ruanzhi]

@eleinadani Hello, I would like to ask why the following links appear: https://github.com/oracle/graaljs/issues/418