assemblyscript icon indicating copy to clipboard operation
assemblyscript copied to clipboard

Prevent webpack from detecting calls to `require` from `dyrequire`

Open AurelienRibon opened this issue 4 years ago • 7 comments

ISSUE

Webpack is used to generate the bundle in dist/ folder, that is included in the npm package. However, when users install the package in their project npm install assemblyscript, include ASC with const asc = require('assemblyscript/dist/asc.js'), and try to bundle their own project with Webpack, there are multiple warnings emitted, saying that calls to require are made in a way that prevents static analysis of dependencies.

This is caused by this code:

// Original source code
const dynrequire = typeof __webpack_require__ === "function"
  ? __non_webpack_require__
  : require;

// Generated code from `npm run build:bundle` (without minification)
const dynrequire = true
  ? require // <--- that's our issue
  : 0;

When users call Webpack themselves in their project, it will detect that dynrequire === require (I was surprised it could do that, the constant condition true may help). Thus, that leads to many errors, from ts-node module is not found (we are in a web bundle...), to "require is called in a way that prevents static analysis".

image

SOLUTION

The variable require must not be present in the distributed code when requiring Node-only stuff. __non_webpack_require__ is replaced by webpack with require. Thus it should only be used by leaf projects, not by libraries, else that creates issues in leaf projects. The solution consists in just replacing that __non_webpack_require__ by eval('require'), so that

  • [x] I've read the contributing guidelines

AurelienRibon avatar Feb 16 '21 08:02 AurelienRibon

Interesting, the eval is what we had originally, but then switched. I don't recall exactly anymore why we did that, hmm. @MaxGraey ?

dcodeIO avatar Feb 16 '21 09:02 dcodeIO

I saw that in dist/sdk.js, you expose asc as require('assemblyscript/cli/asc.js'), for the Node version. Does this mean that dist/asc.js is only meant to be used by browsers? If so, it may even be better to replace __non_webpack_require__ by a throw, because no code path should try to require ts-node in the browser.

In the meantime, this solution works as well, as eval('require') will return undefined, which will throw when called as a function. And this has the benefit of letting Node users still require dist/asc.js, instead of cli/asc.js.

AurelienRibon avatar Feb 16 '21 09:02 AurelienRibon

Yes, dist/asc.js originated as a browser-only build of the compiler. I'd not be opposed to make it more universal, though, if it can be done without breakage.

dcodeIO avatar Feb 16 '21 10:02 dcodeIO

The current dist/asc.js should be useable from the CLI, on master and on this branch too, as it still tries to require ts-node and inject stuff.

That aside, not being able to use dist/asc.js from a Webpack project is an issue, as there are few production grade projects that import dependencies without a bundler. Ideally, asc.js should be decoupled between a lib agnostic from any Node requirement (and that would be exposed for browsers), and a wrapper that deals with ts-node and all that stuff. But one thing at a time :-)

AurelienRibon avatar Feb 16 '21 12:02 AurelienRibon

More background: if webpack is able to infer that dynrequire === require, it is because of the minifier:

// Original source code
const dynrequire = typeof __webpack_require__ === "function"
  ? __non_webpack_require__
  : require;

// Generated code from `npm run build:bundle` (without minification)
const dynrequire = true
  ? require
  : 0

// Generated code with minification
d=require // `d` is dynrequire

Webpack does not resolve static conditions, but it can resolve variable reassignment. I will find a way. It's important that asc.js can be integrated in Webpack bundles.

AurelienRibon avatar Feb 17 '21 14:02 AurelienRibon

Related to this issue, asc.js cannot be required on Observable.

Another solution I though of was to generate two versions of asc.js, one for the CLI, and one just for the browser. Basically, what people want to do with a browser version of asc is just to be able to run compileString and get a WASM binary, nothing more. All this is possible with the assemblyscript.js file, but it's not easy to figure out in which order to call methods (build a program, inject lib strings, inject user strings, parse all that, instantiate the program, compile). A demo proof of concept of such "easy compile from browser" would be awesome :)

image

AurelienRibon avatar Feb 23 '21 13:02 AurelienRibon

The roadmap I have in mind currently is about:

  • Get an ESM build of binaryen.js going
  • Get an ESM build of long.js going (trivial)
  • Transition asc.js to ESM
  • Make an ESM bundle of everything, and transpile to an ES5 module in addition

dcodeIO avatar Feb 23 '21 13:02 dcodeIO

AS already ports to ESM, We can close this PR.

HerrCai0907 avatar Sep 28 '23 01:09 HerrCai0907