[dart2js] Guarantees around deferred loading & prefetching

[mkustermann]

It seems dart2js fine granular splitting of code for deferred loading will not ensure that a await deferred.loadLibrary() is actually loading code for that library.

Here's a reproduction

// main.dart
import 'def.dart' deferred as D;
import 'def1.dart' deferred as D1;
import 'def2.dart' deferred as D2;

main() async {
  // This will not load anything from `def.dart`
  await D.loadLibrary();

  if (int.parse('1') == 1) {
    // This will load parts of `def.dart` that `D1` needs
    await D1.loadLibrary();
    await D1.deferredDef1();
    // This will load other parts of `def.dart` that `D2` needs.
    await D2.loadLibrary();
    await D2.deferredDef2();
  }
}

// def.dart
Future deferredDef() async { // <-- not used at all
  print("deferredDef");
}

void deferredDef_2() {  // <-- used by def2.dart
  print("deferredDef_2");
}

void deferredDef_1() {  // <-- used by def1.dart
  print("deferredDef_1");
}

// def1.dart
import 'def.dart' deferred as D;

Future deferredDef1() async {
  print("deferredDef1");
  await D.loadLibrary();
  D.deferredDef_1();
}

// def2.dart
import 'def.dart' deferred as D;

Future deferredDef2() async {
  print("deferredDef2");
  await D.loadLibrary();
  D.deferredDef_2();
}

% dart compile js -O4 --deferred-map=main.deferred.json --no-minify -o main.js main.dart
Compiled 10,267,481 input bytes (5,248,573 characters source) to 202,515 characters JavaScript in 0.56 seconds

main.deferred.json

{
  "_comment": "This mapping shows which compiled `.js` files are needed for a given deferred library import.",
  "main.dart": {
    "name": "<unnamed>",
    "imports": {
      "D": [],
      "D1": [
        "main.js_2.part.js",
        "main.js_3.part.js"
      ],
      "D2": [
        "main.js_2.part.js",
        "main.js_5.part.js"
      ]
    },
    "importPrefixToLoadId": {
      "D": "D",
      "D1": "D1",
      "D2": "D2"
    }
  },
  "def1.dart": {
    "name": "<unnamed>",
    "imports": {
      "D.1": [
        "main.js_2.part.js",
        "main.js_1.part.js"
      ]
    },
    "importPrefixToLoadId": {
      "D": "D.1"
    }
  },
  "def2.dart": {
    "name": "<unnamed>",
    "imports": {
      "D.2": [
        "main.js_2.part.js",
        "main.js_4.part.js"
      ]
    },
    "importPrefixToLoadId": {
      "D": "D.2"
    }
  }
}

As we can see here when main.dart is await D.loadLibrary() it has an empty array, so it won't load anything.

This optimization is correct in the sense that the await D.loadLibrary() is actually a NOP and can be optimized away because a) main.dart doesn't access anything from D/def.dart b) any other library that accesses something from D/def.dart will have to also perform first a await D.loadLibrary() itself, ensuring the parts it needs are loaded

The consequence of this is that this mechanism cannot be used to pre-fetch deferred modules - at least not the way done in this example.

If one wants to prefetch all of D/def.dart then one needs to call a top-level method in every library that imports def.dart and that top level library has to issue a await D.loadLibrary(). That will ensure all parts will be loaded.