Lack of documentation for migration from js to js_interop

[felix-ht]

With the recent beta release of wasm for flutter i wanted to try compling my application to the new target. As (somewhat) expected i ran into some js interop issues.

I tried to switch to the new package import 'dart:js_interop'; but struggled to get it working as there seems to be very litte documentation on how to do the migration.

Old code

@JS('mapboxgl')
library mapboxgl.interop.ui.control.navigation_control;

import 'package:js/js.dart';
import 'package:mapbox_gl_dart/src/interop/interop.dart';

@JS()
@anonymous
class NavigationControlOptionsJsImpl {
  external bool get showCompass;
  external bool get showZoom;
  external bool get visualizePitch;

  external factory NavigationControlOptionsJsImpl({
    bool? showCompass,
    bool? showZoom,
    bool? visualizePitch,
  });
}

@JS('NavigationControl')
class NavigationControlJsImpl {
  external NavigationControlOptionsJsImpl get options;

  external factory NavigationControlJsImpl(
      NavigationControlOptionsJsImpl options);

  external onAdd(MapboxMapJsImpl map);

  external onRemove();
}

Diff

diff --git a/lib/src/interop/ui/control/navigation_control_interop.dart b/lib/src/interop/ui/control/navigation_control_interop.dart
index 985a203..0257ce2 100644
--- a/lib/src/interop/ui/control/navigation_control_interop.dart
+++ b/lib/src/interop/ui/control/navigation_control_interop.dart
@@ -1,7 +1,7 @@
 @JS('mapboxgl')
 library mapboxgl.interop.ui.control.navigation_control;
 
-import 'package:js/js.dart';
+import 'dart:js_interop';
 import 'package:mapbox_gl_dart/src/interop/interop.dart';
 
 @JS()

The error i am getting is Error: The '@JS' annotation from 'dart:js_interop' can only be used for static interop, either through extension types or '@staticInterop' classes.

I found some documentation in https://dart.dev/interop/js-interop/usage

e.g.

import 'dart:js_interop';

@JS('Date')
extension type JSDate._(JSObject _) implements JSObject {
  external JSDate();

  external static int now();
}

Before converting all the code in my dart js warper i would like to be sure that this is the correct approach. It would be great if you could also provide some concrete side by side examples of old js vs new js_interop code in the documentation.

Config

Flutter 3.21.0-1.0.pre.2 • channel beta • https://github.com/flutter/flutter.git
Framework • revision c398442c35 (3 weeks ago) • 2024-03-12 22:26:24 -0700
Engine • revision 0d4f78c952
Tools • Dart 3.4.0 (build 3.4.0-190.1.beta) • DevTools 2.33.1

[kevmoo]

CC @srujzs @MaryaBelanger

[duck-dev-go]

Is there any update on this? It seems nearly impossible to use wasm without more documentation on this.

[srujzs]

I think Marya has been working on an interop tutorial first and then this, but feel free to ask me questions in the meantime on how to port specific lines of code.

The general model is similar to package:js and we dive into some of the major differences here: https://dart.dev/interop/js-interop/past-js-interop#package-js.

---

Looking at the OP edits:

Error: The '@JS' annotation from 'dart:js_interop' can only be used for static interop, either through extension types or '@staticInterop' classes.

This is mildly covered on that webpage, but there are two @JS annotations - one from package:js (which is meant for package:js classes and can be unsound) and one from dart:js_interop (which is meant for extension types and is sound).

Before converting all the code in my dart js warper i would like to be sure that this is the correct approach.

In general, you should move code from classes to extension types on JSObject. external members can be more or less copied over to the extension type, with the caveat that the types you can use on such a member are restricted: https://dart.dev/interop/js-interop/js-types#requirements-on-external-declarations-and-function-tojs. The above code seems to only use primitives and interop types, so the migration would be rather simple (assuming MapboxMapJsImpl is also migrated to an interop extension type):

@JS('mapboxgl')
library mapboxgl.interop.ui.control.navigation_control;

import 'dart:js_interop';

import 'package:mapbox_gl_dart/src/interop/interop.dart';

// No need for `@JS` if not renaming, and no need for `@anonymous` as you can
// write an external constructor with named parameters directly (as shown below).
extension type NavigationControlOptionsJsImpl._(JSObject _) implements JSObject {
  external bool get showCompass;
  external bool get showZoom;
  external bool get visualizePitch;

  external factory NavigationControlOptionsJsImpl({
    bool? showCompass,
    bool? showZoom,
    bool? visualizePitch,
  });
}

@JS('NavigationControl')
extension type NavigationControlJsImpl._(JSObject _) implements JSObject {
  external NavigationControlOptionsJsImpl get options;

  external factory NavigationControlJsImpl(
      NavigationControlOptionsJsImpl options);

  external onAdd(MapboxMapJsImpl map);

  external onRemove();
}

[duck-dev-go]

Following the example with extension types, I can now no longer implement my own interface.

extension type JavascriptXHROptions._(JSObject _) implements JSObject, XHROptions

Which will give me an error saying XHROptions is not a supertype of JSObject. But I use my own interface for dependency inversion throughout my code. How can I with this new approach implement my own interfaces so my other projects won't break?

[srujzs]

What is XHROptions here? A package:js class? If so, the error makes sense: the representation type is not a subtype of XHROptions. Interop extension types are not supposed to compose package:js types so changing the representation type to XHROptions won't work either (you'll get a compile-time error saying JavascriptXHROptions is not an interop extension type now).

Instead, if you want to still reuse the interface of XHROptions, use a cast:

extension type JavascriptXHROptions._(JSObject _) implements JSObject {}
...
final javascriptXHROptions = ...;
(javascriptXHROptions as XHROptions).someMethod();

Note that package:js can't be used anywhere in your program when compiling to Wasm. If you make XHROptions an interop extension type, you can implement it like you have.

[duck-dev-go]

XHROptions is my own custom abstract class that serves as an interface only.

abstract class XHROptions {
  XHROptions({
    required String method,
    required Map<String, String> headers,
    required bool withCredentials,
  });

  String get method;
  set method(String value);

  Map<String, String> get headers;
  set headers(Map<String, String> value);

  bool get withCredentials;
  set withCredentials(bool value);
}

I just took it as an example as I have many of these interfaces. But my point was that before with package:js I was able to implement that interface like so

@JS()
@anonymous
class JavascriptXHROptions implements XHROptions {

But with the new approach

extension type JavascriptXHROptions._(JSObject _) implements JSObject, XHROptions

This now seems like it's no longer possible. Because XHROptions is not a super type of JSObject, and I don't want it to be. I want the interace to be separated from any js related coupling. Instead I use it for dependency inversion in my other packages to avoid them being tightly coupled to any web or js related package. But now it seems I cannot implement it anymore with this new approach.

So how do I migrate my package js code so that it doesn't break all my other packages because I cannot use the interfaces anymore.

[srujzs]

I see, you're just using it as an interface. Extension types by design do not allow virtual/dynamic dispatch, so you can't have an interop extension type implement an unrelated Dart class. One option to make this work (that will be slower) is a wrapper class:

extension type JavascriptXHROptions._(JSObject _) implements JSObject {
  // whatever interop members you might need
  JavascriptXHROptions({String method});
  external String method;
}

class JavascriptXHROptionsImpl implements XHROptions {
  late final JavaScriptXHROptions _options;

  JavascriptXHROptionsImpl({required String method}) {
    _options = JavascriptXHROptions(method);
  }

  String get method => _options.method;
  set method(String value) => _options.method = value;
}

[duck-dev-go]

Thanks for the example.

The irony, the very thing extension types would solve is the very thing they can't solve in a way.

IMO most people would never couple their project tightly to javascript code as flutter is a multi platform framework and in general dependency inversion is very commonly used in the flutter ecosystem. So the new extension type mostly just adds more boilerplate code for framework users. Where the old way of doing it with the js package likely has similar performance to this approach but less boilerplate code.

I'm a flutter user btw.

[srujzs]

Maybe. :) package:js classes disallowed you from having non-external members, so if you needed to do anything besides just directly call the JS member (with the same name as the interface API since you couldn't do renaming), you'd likely need to use a wrapper class anyways.

I've mostly seen that approach with more granular APIs that call a number of JS members and handle the results, so you need a wrapper class regardless in that case. However, in the case where you just need a set of options, like XHROptions, I can definitely see it being worse to use extension types.

[duck-dev-go]

In most cases we don't need to have non-external members on a JS class. Also wouldn't you be able to write extensions for the package:js classes, it seems to work for me? I'm still not seeing the advantage of using an extension type for this, but perhaps I'm overlooking something.

[srujzs]

Sorry, my comment might have been confusing - it was in reference to specifically implementing a shared interface.

Also wouldn't you be able to write extensions for the package:js classes, it seems to work for me?

Right, but that's the same as extension types in that the non-external methods wouldn't have virtual dispatch since they're extension methods. So, they couldn't be used to implement an interface, but if the external JS members directly implement the interface you need, you don't need the non-external members, and therefore the package:js class would be better.

I'm still not seeing the advantage of using an extension type for this, but perhaps I'm overlooking something.

For dependency inversion, there isn't an advantage. I was mostly pointing out that it might not be worse than package:js classes depending on the use case.

[felix-ht]

@srujzs your example is exactly what i ended up doing

Converting over 8200 lines of pure interface code took like two days, so its very much manageable.

The bigger issues i faced showed up as i tired to run the code after it was complied to wasm as suddenly the JS - dart interface becomes much stricter so any existing issue in the interface suddely start throwing errors. If you just complie to JS many issues don't show up. Managed to fix them tho.

I also ran into issues with other lib's ablity to compile to wasm, so any futher work was put on hold untill the ecsosystem catches up. Most of these issues i expect to have been resolved by now.

@duck-dev-go Pushed the full project if you need something to compare against:

https://github.com/Ocell-io/mapbox-gl-dart/tree/new-js_interop

[srujzs]

Nice! Yeah, I expect the change in what types are allowed will be the bigger lift than the change to extension types.

The bigger issues i faced showed up as i tired to run the code after it was complied to wasm as suddenly the JS - dart interface becomes much stricter so any existing issue in the interface suddely start throwing errors. If you just complie to JS many issues don't show up. Managed to fix them tho.

Can you elaborate what you mean by this? Do you mean you saw errors going from package:js to dart:js_interop or going from compiling to JS to compiling to Wasm? With dart:js_interop, we should hopefully have the same errors around restricted types regardless of whether you compile to JS or Wasm.

[felix-ht]

Is saw the error after porting to js_interop and had a basic running version with js. Some issues only showed up after i tried to compling to wasm. This was a multistage process. dart2wasm showed issues during compiling, while dart2js complied just fine. After that i also had some runtime errors with dart2wasm that were tricky to debug, as there is no debugger for wasm, and the error message / stacktrace was lacking as there weren't even debug symbols.

These two commit hold the changes required to get wasm running.

https://github.com/Ocell-io/mapbox-gl-dart/commit/e811c3d4db82b7d2299c2f620f08a1f8f28ec593

https://github.com/Ocell-io/mapbox-gl-dart/commit/d7366bab7135d0de7c17b8fe83b95fc0f5470d03

[felix-ht]

Looking at the code again it seems that all (most?) issues where with the js object container layer not with the interop itself.

[srujzs]

Ah I see, this is partially a change in imports from dart:js_util to dart:js_interop. We don't have any errors on the JS backends telling users to use dart:js_interop instead because the old JS interop libraries are not deprecated yet. There's a lint request to add that and in turn encourage migration, though. Thanks for sharing!

[pedromassango]

This is concerning, when the team recommends migrating from on package to another and no migration guide is provided, ideally we shouldn't assume that everyone knows how to do it :(

This really breaks the UX of using Dart

[patniemeyer]

The documentation on js_interop refers to globalThis which maybe was replaced by globalContext, but in either case does not seem to allow dynamic property setting as shown in the example. Can globalContext be used for anything directly?

[srujzs]

The use of globalThis in the documentation (e.g. here) is within the scope of a JS program to illustrate setting and referring to properties. globalContext is the rough Dart equivalent to globalThis. I say rough because it may be self or some other slightly different way to refer to the global scope due to legacy reasons, but practically speaking, it'll make no difference in the vast majority of cases and you can treat it like globalThis. So, you can use it however you would with globalThis.

Dynamic property setting can be handled using dart:js_interop_unsafe. Preferably, you should use static interfaces where possible, but we still allow you to access, set, and call arbitrary properties.

[patniemeyer]

For anyone else who would benefit from a more complete example: I have just implemented a full (typescript) scripting extension mechanism in my Flutter for web project using js_interop. It involves bindings for not-completely-trivial types and function calling both ways as well as evaluation of scripts in the browser environment via calling out to eval. Code is on github

[pmathew92]

What would be the dart:js_interop equivalent of objectKeys(Object? object ) and newObject<T> method from the dart:js_util library ? objectKeys method allows me to iterate through a dart object keys which I use at multiple places in my codebase. Now migrating to wasm this causes a problem as I can't seem to find any alternate for the same . Could someone help here ?

[srujzs]

Object.keys is a static method available globally, so you can just use interop:

@JS('Object.keys')
external JSArray<JSString> keys(JSObject o);

The JSObject() constructor replaces the newObject method. You could write your own interop extension type for Object with an external constructor to do this as well, but it's not worth the extra code.

[patniemeyer]

Object.keys is a static method available globally, so you can just use interop:

@JS('Object.keys') external JSArray<JSString> keys(JSObject o);

The JSObject() constructor replaces the newObject method. You could write your own interop extension type for Object with an external constructor to do this as well, but it's not worth the extra code.

This is the trick - If you need to do things dynamically just wrap the relevant JS API so that you can invoke it statically... In my project above I wrapped JS eval().

[pmathew92]

Thank you @srujzs and @patniemeyer

[pmathew92]

Hi @patniemeyer @srujzs , Sorry i have another problem I am running into I have a class which I wanted to convert to js_interop type but having a bit trouble

@JS()
class Client {
  external Client(final Client options);
  external Future<void> loginWithRedirect([final RedirectLoginOptions options]);
  external Future<void> loginWithPopup(
      [final PopupLoginOptions? options, final PopupConfigOptions? config]);
  external Future<void> handleRedirectCallback([final String? url]);
  external Future<void> checkSession();
  external Future<WebCredentials> getTokenSilently(
      [final GetTokenSilentlyOptions? options]);
  external Future<bool> isAuthenticated();
  external Future<void> logout([final LogoutOptions? logoutParams]);
}

I converted it to a extension type

@JS()
extension type Client._ (JSObject _ ) implements JSObject {
  external Client(final Client options);
  external JSPromise<JSAny> loginWithRedirect([final RedirectLoginOptions options]);
  external JSPromise<JSAny> loginWithPopup(
      [final PopupLoginOptions? options, final PopupConfigOptions? config]);
  external JSPromise<JSAny> handleRedirectCallback([final String? url]);
  external JSPromise<JSAny> checkSession();
  external JSPromise<WebCredentials> getTokenSilently(
      [final GetTokenSilentlyOptions? options]);
  external JSPromise<JSBoolean> isAuthenticated();
  external JSPromise<JSAny> logout([final LogoutOptions? logoutParams]);
}

Is the above conversion type correct ? Althogh it compiles it leads to run time errors like Type 'Null' is not a subtype of type 'JSValue' in type cast . Also I have methods which is of type Future but there isn't any thing that corresponds exactly to this . What should I use for methods that return Future ?

[srujzs]

You are correct in using JSPromise (I'm surprised using Future to represent promises with the old interop worked). You'll likely need to change the generic type for the JSPromises to JSAny? instead. If resolving these promises result in undefined, that gets converted to Dart null, and therefore you'll see the type error, as a null is not a JSAny (internally a JSValue on dart2wasm).

[MichealReed]

Some workloads are not returned instantly and in languages like C we cannot mark them explicitly async without an additional dependency on embind. From my understanding emscripten recommends use of ccall, but I do not see a way ccall can be used with the new interop structure given we cannot set ccall's return type as JSPromise as

https://emscripten.org/docs/porting/asyncify.html#usage-with-ccall

Error: External JS interop member contains invalid types in its function signature:
'JSPromise<JSAny?> Function(String, String, *List<String>*, *List<dynamic>*, *Map<dynamic, dynamic>*)'.
Use one of these valid types instead: JS types from 'dart:js_interop', ExternalDartReference, void, bool, num, double, int, String, extension types that   
erase to one of these types, '@staticInterop' types, 'dart:html' types when compiling to JS, or a type parameter that is a subtype of a valid non-primitive
type.
external JSPromise _ccall(
    String name, String returnType, List<String> argTypes, List args, Map opts);

is encountered on compile.

With the old approach we could easily get a Dart awaitable Future with

Future<int> function() async =>
    promiseToFuture(_ccall("function", "number", [],
        [], {"async": true}));

The other approach emscripten recommends is to add -sASYNCIFY_IMPORTS=function because he compiler doesn’t statically know that function is asynchronous when coming from c.

https://emscripten.org/docs/porting/asyncify.html#ways-to-use-asyncify-apis-in-older-engines

Using this does not result in an awaitable object, but instead dart resolves to null. With

DartError: TypeError: null: type 'Null' is not a subtype of type 'JSObject'

Is EMBind required with the new interop for older engines? It seems more inclusive for dart to allow or even wrap ccall via the interop or wasm library or recognize the asyncify exports somehow

https://emscripten.org/docs/porting/asyncify.html#usage-with-embind

[srujzs]

You can and should use JSPromise. The error has some highlighting to indicate this (types wrapped in **) but the issues are due to the List and Map parameters, not the JSPromise return value. You need to accept JSArrays and JSObjects instead. You can use jsify to convert simple lists and maps (with String keys) to arrays and objects.

[MichealReed]

You can and should use JSPromise. The error has some highlighting to indicate this (types wrapped in **) but the issues are due to the List and Map parameters, not the JSPromise return value. You need to accept JSArrays and JSObjects instead. You can use jsify to convert simple lists and maps (with String keys) to arrays and objects.

Here is some example code for another attempt to use ccall. The externally defined _mgpuReadBufferSync works but takes longer to complete than calls after it like print, so the data is wrong.

// Working but slow completion
@JS('_mgpuReadBufferSync')
external void _mgpuReadBufferSync(
    MGPUBuffer buffer, JSNumber outputDataPtr, JSNumber size);

Edit: this seems to work, thanks for your help!

@JS('ccall')
external JSPromise _ccall(JSString name, JSString returnType, JSArray argTypes,
    JSArray args, JSObject opts);

    await ccall(
            "mgpuReadBufferSync".toJS,
            "number".toJS,
            ["number".toJS, "number".toJS, "number".toJS].toJS,
            [buffer, ptr, size.toJS].toJS,
            {"async".toJS: "true".toJS}.toJSBox)
        .toDart;

or using js_interop_utils https://pub.dev/packages/js_interop_utils

await _ccall(
            "_mgpuReadBufferSync".toJS,
            "number".toJS,
            ["number", "number", "number"].toJSDeep,
            [buffer, ptr, size.toJS].toJSDeep,
            {"async": true}.toJSDeep)
        .toDart;

It would be helpful if the docs included common examples for converting lists and maps as well as foreign functions that rely on these for arguments.

[pmathew92]

Hi, What would be the best way to modify the test class when replacing a class with Extension types

I have a Mock class created for testing purpose which implements a class . This is called by my actual test classes for mocking purpose. But after the changes, it is throwing Classes and mixins can only implement other classes and mixins.

Here is a sample of how I am using this

_FakeClient_0(
  Object parent,
  Invocation parentInvocation,
) : super(
        parent,
        parentInvocation,
      );
}```

I have looked into the https://dart.dev/interop/js-interop/mock but can't seem to figure out how to use this in my current implementation