Refactor Inquire so it can resolve modules in Browsers with default CPS

[seanlangbrown]

inquire() cannot resolve any modules in browsers that do not allow unsafe-eval

inquire() will always return null in browsers with default CPS settings, even if the requested module is present. This PR fixes that issue while maintaining all desired bundling behavior for inquire.

Problem

While investigating https://github.com/protobufjs/protobuf.js/issues/1483 I found that browsers are blocking the execution of eval() in inquire(). This repl reproduces and investigates the issue.

The purpose of inquire is to require a module if it is already available in the environment, but "hide" the module from bundlers so that it is not included as a dependency. To do this, inquire js used eval() and regex as a workaround so that bundlers do not notice the require call during static code analysis:

function inquire(moduleName) {
    try {
        var mod = eval("quire".replace(/^/,"re"))(moduleName); // eslint-disable-line no-eval
        if (mod && (mod.length || Object.keys(mod).length))
            return mod;
    } catch (e) {} // eslint-disable-line no-empty
    return null;
}

Unfortunately, today's browsers have CPS settings that block eval() by default, so inquire() cannot require any modules at all (including modules that are available in the environment). Changing the defaults so that 'unsafe-eval' is allowed affects the entire page and is not recommended for security. Many less experienced web developers using protobufJS see the warning messages in the browser console and think that they must allow unsafe-eval to use protobufjs, for example https://github.com/protobufjs/protobuf.js/issues/593, https://github.com/protobufjs/protobuf.js/issues/1483.

We can show that inquire() does not work with CPS defaults by simulating the browser environment like this:

function runInquireInBrowser(moduleName) {
  var blockedEval = function(){throw Error("Unsafe-Eval is blocked")};
  var browserCSPDefault = { eval: blockedEval, "Function": blockedEval };
  
  with (browserCSPDefault) {
      try {
          var mod = eval("quire".replace(/^/,"re"))(moduleName); // EXCEPTION: "Unsafe-Eval is blocked"
          if (mod && (mod.length || Object.keys(mod).length))
              return mod;
      } catch (e) {} // EXCEPTION is caught
        return null; // always returns null
  }
}
const result = runInquireInBrowser('fs');

We can reason that result will always be null.

Solution

Using eval() for this made sense 10+ years ago. There is now an accepted "standard" way of configuring "externals" so dynamic requires can be made while excluding those modules from the bundles. It is implemented by all major bundlers including webpack, browserify (gulp), rollup, and possibly others. By using this standard we can exclude inquired modules (long and buffer) from protobufjs distributions.

There is also now a way to instruct bundlers not to include dependencies of protobufJS distributions: https://github.com/defunctzombie/package-browser-field-spec#ignore-a-module. This can be used to prevent webpack from bundling long with a website that depends on protobufjs.

Implementation

• New test cases for inquire() in a simulated browser environment with eval blocked
• Removed eval() from inquire
• Added all modules loaded with inquire() to the broswerify config "externals" to prevent protobufJS from including inquired modules in it's bundle.
• Added all modules loaded with inquire() to the protobufJS package.json "browser" field so that any project that uses protobufJS and a bundler will not add inquired modules to it's bundle.

[mehcode]

Is there anything we can do here to merge this in and cut a release? This is a pretty important change for use of protobufjs in browsers.

@tinder-seanlang-brown Did you end up publishing a fork to npm somewhere?

[yvele]

Is this PR also aimed to fix the codegen problem regarding CSP? https://github.com/protobufjs/protobuf.js/issues/1483#issue-687679653

[paralin]

FYI Due to this issue esbuild throws warnings about using Eval, one per instance of protobufjs being imported.

[bhollis]

We worked around this at bundle generation time using a Webpack alias:

    alias: {
      /**
       * https://github.com/protobufjs/protobuf.js/issues/997 The original inquire module contains
       * usage of eval, which triggers a CSP violation. Currently we always generates static code
       * for protos, so there is no need for any reflection, thus we don't need inquire to work.
       */
      "@protobufjs/inquire": path.resolve(__dirname, "src", "patches", "inquire.js"),
    },

Where our replacement inquire.js is:

"use strict";
module.exports = inquire;

function inquire() {
  return null;
}