[Proposal]: Support for Workers

[ivanbuhov]

Support for Workers

For Android solution please check - https://github.com/NativeScript/android-runtime/issues/532

Description

General guidelines for the Workers implementation effort and specification in the context of NativeScript. We have an existing issue here, but the purpose here is to show the road map that the team intends to follow in developing the Workers functionality. The general scenarios we want to support are listed at the bottom.

Limitations

In NativeScript we don’t need to implement all details of the web workers specification, because some of these details are only related to the browser, and won’t have any meaning in the context of a NativeScript application.

List of features we decided not to support:

• NativeScript Worker class doesn't implement the EventTarget interface
• Workers Global Object doesn't implement EventTarget, WindowTimers, WindowBase64, WindowEventHandlers interfaces.
• Workers Global Object has no the following properties: caches, navigator, location, ononline, onoffline, onlanguagechange.

The features and syntax that will follow has the purpose of describing the adoption of the web workers specification in NativeScript. In this document we will describe and list what we intend to support.

Guidelines from the specification to follow

context

Notice that onmessage and postMessage() need to be hung off the Worker object when used in the main script thread, but not when used in the worker. This is because, inside the worker, the worker is effectively the global scope.

messaging specifics

Data passed between the main thread and workers is copied, not shared. Objects are serialized as they're handed to the worker, and subsequently, deserialized on the other end. The page and worker do not share the same instance, so the end result is that a duplicate is created on each end. Most browsers implement this feature as structured cloning. The structured cloning algorithm can accept JSON and a few things that JSON can't — like circular references.

messaging transfer ownership

Passing data by transferring ownership (transferable objects): in nativescript we can try to pass native objects by transferring ownership on them.

onerror spec

The error event has the following three fields that are of interest: message: A human-readable error message. filename: The name of the script file in which the error occurred. lineno: The line number of the script file on which the error occurred.

Syntax

The following example will be the general syntax to use when working with Workers. Syntax is based on web worker specification.

<app_name>/app/main.js

// create new worker
var myWorker = new Worker("worker.js");

// on worker error handler
myWorker.onerror = function (e) {
 //do on worker error
}

// receive messages from worker
myWorker.onmessage = function (e) {
  result.textContent = e.data;
}
....
// send messages to worker
myWorker.postMessage("message will be sent to worker");

//The worker thread is killed immediately without an opportunity to complete its operations or clean up after itself.
myWorker.terminate();

myWorker.onclose = function () {
//on close (clean up)
}

---

<app_name>/app/worker.js

onmessage = function(e) {
  postMessage(e.data); // worker -> main
}

//worker closes it self
close();

Supported APIs:

Worker Object:

• [x] Worker.onerror
• [x] Worker.onmessage
• [x] Worker.postMessage()
• [x] Worker.terminate()

Worker Global Object:

• [x] WorkerGlobalObject.self
• [x] WorkerGlobalObject.onmessage
• [x] WorkerGlobalObject.postMessage()
• [x] WorkerGlobalObject.onerror
• [x] WorkerGlobalObject.onclose
• [x] WorkerGlobalObject.close()
• [x] ~~WorkerGlobalObject.importScripts()~~ We decided not to support it. Single require function is enough.

Implementation steps:

• [x] Run pure javascript file on new thread
  • [x] Initialize the iOS runtime on new thread
  • [x] Create a run loop on the newly created thread and schedule the runtime on it
• [x] Enable require of pure javascript files from new thread
• [x] Implement error handling per thread
  • [x] Try/catch should work like it does in the main thread
  • [x] When error is not caught, call onerror handler of the worker object on the main thread
  • [x] Handle UI APIs on worker thread (throw error if possible) - We have no special handling for this case. However, CoreAnimation APIs throw exception if UI operation is executed on a background thread. The exception is thrown in CFRunLoopObserver installed by the CoreAnimation framework. The exception is catched by the system and the runloop is not stopped. We can't place our code between the exception throwing and the exception catching, so we can't handle it in a custom way. However, the system logs a warning in the system log which will be displayed in the CLI output.
• [x] Call native non-ui APIs from javascript file running on thread
  • [x] iOS runtime tests pass on worker thread (except for this)
• [ ] Implement messaging API to communicate between threads (main -> worker, worker->main)
  • [x] Simple object - JSON stringifiable objects
  • [ ] Simple object and an ArrayBuffer as a second parameter
  • [ ] Native objects (change of ownership instead of copy)
• [ ] Enable tns-module support (will extend later)
• [ ] Enable nested workers support - should be easy for implementation
• [ ] Debugging
  • [ ] Resarch what should be done on the backend side in order to support worker debugging
  • [ ] Debugging support in NativeScriptAppInspector client
  • [ ] Debugging support in VSCode
• [x] Add tests
• [x] Update TypeScript definitions -> PR: https://github.com/NativeScript/NativeScript/pull/2900
• [x] Update documentation -> PR: https://github.com/NativeScript/docs/pull/538

[speigg]

Will the Worker be able to leverage FTL JIT compilation ? If not, it loses a lot of value for me. It should be possible to leverage FTL JIT compilation if the worker is implemented via a WKWebView, rather than a JSContext.

[ivanbuhov]

Hi @speigg.

Worker threads will not take advantage of the FTL JIT but keep in mind that the same is valid for the main JS thread too. The lack of FTL JIT compilation is due to iOS restriction which forbids allocating writable and executable memory.

Implementing Workers via WKWebView means that we have to use the built-in JavaScriptCore which comes with too many limitations. In the early days of NativeScript we evaluated both options - built-in JSC vs custom JSC build and we strongly believe that using a custom build of JSC is the right approach. Many of the existing iOS runtime features require using a custom JSC build.

If not, it loses a lot of value for me

Can you be more specific on this? It would be great if you provide some use cases in which a lot of value is lost due to the lack of FTL JIT.

[speigg]

I'm decrypting messages using openpgpjs. In the main JS thread, this can pause the UI for up to 8 seconds. I have it now working in a hacked together web worker with some basic structured cloning support (just serializing/deserializing typed arrays and Date objects, not dealing with cyclic references or any other web worker features) using a WkWebView. This cuts down the decryption time to about a second. A second I can deal with, but for my app 8 seconds is way too long, as the decryption happens in response to a user action and the user would be forced to wait while things "load" for 8 seconds or more. The other nice thing about WKWebView as a worker (for me) is that openpgp can automatically leverage the built in web crypto APIs.

[speigg]

Perhaps the limitations you mention with the built-in JSC were unsuitable for the main JS Thread (in order to support performant access to native libraries, I'm guessing?), and these limitations may not be as much of a concern for web worker use cases. I think it would be a great balance to have the main thread be "slower" but with quick interop between native and JS, while a web worker runs JS in a highly optimized manner but with slow interop with native libraries. Or perhaps even allowing a developer to choose how the worker should run: optimizing for interop, or for raw JS performance.

[ivanbuhov]

Hi @speigg

Having a custom JSC build allow us to use the C++ API of JSC which brings many of the runtime features. Some of them are:

• Access to more native APIs
• The ability to extend native classes with TypeScript
• Deeper integration between native and JavaScript e.g. the ArrayBuffer <-> NSData marshaling
• Timeline profiler
• GC control APIs
• LiveEdit

Other benefits of using a custom JSC build are:

• ES6 features
• Predictability and compatibility - right now an app built with our custom build when ran on iOS 8, 9 (and soon 10) would have the exact same JavaScript features and behavior, whereas an app which depends on the built-in WKWebView would be limited by the JavaScript features of its lowest deployment target.

Possibly, we can add more bullets to the list if we ever start working on alternative solution (e.g. WKWebView based bridge).

To be honest, I like the idea of allowing a developer to choose how the worker should run but even if we are still fine to give up some of the runtime features in order to support WKWebView based workers, this would still be a very huge effort. Most of our codebase can't be reused as is. The task would be near to implementing and supporting a second bridge. We need very good reasons (e.g. much more use cases and currently unsolvable scenarios even by using a native libs) to take this path.

On your particular case: What about using a native library for the decryption part?

[speigg]

Yes, a native library would work too (though I would have to create a plugin that wraps an appropriate library for each platform).

[NathanaelA]

@speigg - My NS-WebWorker plugin actually wraps the WKWebView... We don't need the Team to do this; it already exists. However, the issue with it is that you have no access to the native api's which is the tradeoff...

[roblav96]

How does one go about experimenting with the preview channel of the WebWorkers?

[jasssonpet]

If you want to try them out now, you'll have to checkout this branch and create an Xcode project as instructed in the readme. After that open the HelloWorld target and create some JS files in the app folder.

[roblav96]

@jasssonpet Sounds good. I'm gonna play around with this 😄

[farfromrefug]

Is there any plan to move forward on this? I would love to be able to pass native object.

[mbektchiev]

Hi @farfromrefug, we haven't planned on new worker features for the immediate future.

But we've managed to devise a workaround for passing native instances to workers by passing their pointers instead. Taking the unit test from the above pull request for example you can achieve it like this:

var worker = new Worker("./EvalWorker.js");
var nativeDict = NSDictionary.dictionaryWithObjectForKey("value", "key");
var message = {
    value: { dictionaryPtr: interop.handleof(nativeDict).toNumber() }
};
// increase reference count to account for `dictionaryPtr`
nativeDict.retain(); 
worker.postMessage(message);

In the worker you can retrieve the native object like this:

function pointerTo(type, value) {
    var outerPtr = interop.alloc(interop.sizeof(interop.Pointer));
    var outerRef = new interop.Reference(type, outerPtr);
    outerRef.value = value;
    return outerPtr;
}

function valueFromPointerNumber(type, value) {
    const ptr = new interop.Pointer(value);
    const ptrToPtr = pointerTo(interop.Pointer, ptr);
    const ref = new interop.Reference(type, ptrToPtr);

    return ref.value;
}


onmessage = function(msg) {
    var dict = valueFromPointerNumber(NSDictionary, msg.data.value.dictionaryPtr);
    // decrease reference count
    dict.release();
}

While trying to make it work I discovered a bug in the runtime which is currently preventing the above code to work on 64-bit devices and Simulators. But you can test it for example on an iPhone 4s Simulator ор device.

After we merge the fix it will shortly be available as a @next version of tns-ios. To use it you'll have to install it with tns platform remove ios; tns platform add ios@next.

[farfromrefug]

@mbektchiev thanks a lot! the same technique was proposed on android. Will try it soon to see if it work.

[mbektchiev]

No problem. Tell us if it works out for you after you try it.

I forgot to mention something else which is very important -- to correctly keep the native object alive you'll have to call retain and release manually. I've added it to to example above.

[farfromrefug]

@mbektchiev is the fix already released for your code sample to work? Finally had the time to test this. and in the worker code the dict object is null.

EDIT: it works! will share an example here really soon

[farfromrefug]

@mbektchiev as explaned before i got it to work. Now i am facing another issue. What i am trying to achieve is call a worker method from an Objective-C delegate call made on a background Thread:

• Camera Capture delegate call captureOutput:didOutputSampleBuffer:fromConnection
• right now to send this to a worker i need to capture this in a Nativescript View, notify, then from there postMessage

Right now this is an issue because of memory management, so i have to "clone" the image in didOutputSampleBuffer before sending it to the worker, which is done on UI thread and thus slow. I have to go through the main thread in between the capture background thread and the worker thread.

No real question here, just kind of brainstorming. Does anyone see a way of directly going to the worker?

EDIT: sample code is the demo of the plugin i am working on.

[mbektchiev]

Why do you need to clone it on the main thread. Can't you follow the advice in https://developer.apple.com/documentation/avfoundation/avcapturevideodataoutputsamplebufferdelegate/1385775-captureoutput?language=objc:

If you need to reference the CMSampleBuffer object outside of the scope of this method, you must CFRetain it and then CFRelease it when you are finished with it.

[farfromrefug]

@mbektchiev right now i use OpenCV CvVideoCamera for easy testing/dev

The mat that is passed in processImage is released by CvVideoCamera after that method is done. But in our case the mat is sent asynchronously to the worker. By the time the mat is used by the worker, the mat is released. That's why i need to clone the mat.

[farfromrefug]

@mbektchiev i think there is an issue with the retain, release If i retain in main and release in worker, the objects contained in the NSDictionary are never released If i dont use retain, the dict that i read from the worker is null.

// main

function postMessageToWorker(data) {
            if (isIOS) {
                // the clone makes the UI slow! No solution right now
                const nativeDict = NSMutableDictionary.dictionaryWithObjectForKey(data, 'data');
                const message = {
                    value: { dictionaryPtr: interop.handleof(nativeDict).toNumber() }
                };
                // increase reference count to account for `dictionaryPtr`
                (nativeDict as any).retain();
                worker.postMessage(message);
            }
        }

function postImageToWorker() {
            const folder = knownFolders.currentApp();
            const imageFromLocalFile = fromFile(path.join(folder.path, 'images/test.jpg'));
            const imageSource = new ImageSource();
            imageSource.loadFromBase64(imageFromLocalFile.toBase64String('jpg'));
            const image = imageSource.ios;
            this.postMessageToWorker(image);
}

//worker
equire('globals');
import {  isIOS } from 'tns-core-modules/ui/page/page';

function pointerTo(type, value) {
    const outerPtr = interop.alloc(interop.sizeof(interop.Pointer));
    const outerRef = new interop.Reference(type, outerPtr);
    outerRef.value = value;
    return outerPtr;
}

function valueFromPointerNumber(type, value) {
    const ptr = new interop.Pointer(value);
    const ptrToPtr = pointerTo(interop.Pointer, ptr);
    const ref = new interop.Reference(type, ptrToPtr);

    return ref.value;
}
(global as any).onmessage = function(msg) {
    if (isIOS) {
        const dict = valueFromPointerNumber(NSDictionary, msg.data.value.dictionaryPtr) as NSDictionary<string, any>;
        const data = dict.objectForKey('data');
        // decrease reference count
        (dict as any).release();

    }
};