Using both requireJsDomEnv and Node.js causes an error at jsEnvInput phase

[xerial]

I tried to use jsdom and Node.js at the same time by setting requireJsDomEnv in Test := true with Scala.js 1.0.0 and scalajs-bundler 0.17, but only one of the code using jsdom or Node.js works. Is it possible to use both jsdom and Node.js within the same project?

If I remove the code that are accessing Node.js fs module, the test using jsdom works. If I set requireJsDomEnv in Test := false, the code using Node.js works, but jsdom part fails because window and browser are not properly initialized.

It looks like this node command execution is failing: https://github.com/scalacenter/scalajs-bundler/blob/08cd66ba26716301c09cae5853b42a7cc31e1c05/sbt-scalajs-bundler/src/main/scala-sjs-1.x/scalajsbundler/sbtplugin/Settings.scala#L167

Reproduction

• Checkout https://github.com/wvlet/airframe/pull/950 https://github.com/wvlet/airframe/pull/950/commits/7f5b569034c8172d0a088ea75fdb25fd682a5d11
• Run:

$ ./sbt 
> widgetJS/test

Error message

[error] ModuleNotFoundError: Module not found: Error: Can't resolve 'fs' in '/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test'
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/webpack/lib/Compilation.js:823:10
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/webpack/lib/NormalModuleFactory.js:397:22
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/webpack/lib/NormalModuleFactory.js:130:21
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/webpack/lib/NormalModuleFactory.js:224:22
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/neo-async/async.js:2830:7
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/neo-async/async.js:6877:13
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/webpack/lib/NormalModuleFactory.js:214:25
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:213:14
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error]     at eval (eval at create (/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/UnsafeCachePlugin.js:44:7
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error]     at eval (eval at create (/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error]     at eval (eval at create (/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:27:1)
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:67:43
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error]     at eval (eval at create (/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:28:1)
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/ModuleKindPlugin.js:30:40
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error]     at eval (eval at create (/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error]     at eval (eval at create (/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/forEachBail.js:30:14
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error]     at eval (eval at create (/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/UnsafeCachePlugin.js:44:7
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error]     at eval (eval at create (/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
[error]     at /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/enhanced-resolve/lib/Resolver.js:285:5
[error] resolve 'fs' in '/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test'
[error]   Parsed request is a module
[error]   using description file: /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/package.json (relative path: .)
[error]     Field 'browser' doesn't contain a valid alias configuration
[error]     resolve as module
[error]       /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/node_modules doesn't exist or is not a directory
[error]       /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/node_modules doesn't exist or is not a directory
[error]       /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/node_modules doesn't exist or is not a directory
[error]       /Users/leo/work/git/airframe/airframe-rx-widget/.js/node_modules doesn't exist or is not a directory
[error]       /Users/leo/work/git/airframe/airframe-rx-widget/node_modules doesn't exist or is not a directory
[error]       /Users/leo/work/git/node_modules doesn't exist or is not a directory
[error]       /Users/leo/work/node_modules doesn't exist or is not a directory
[error]       /Users/leo/node_modules doesn't exist or is not a directory
[error]       /Users/node_modules doesn't exist or is not a directory
[error]       /node_modules doesn't exist or is not a directory
[error]       looking for modules in /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules
[error]         using description file: /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/package.json (relative path: ./node_modules)
[error]           Field 'browser' doesn't contain a valid alias configuration
[error]           using description file: /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/package.json (relative path: ./node_modules/fs)
[error]             no extension
[error]               Field 'browser' doesn't contain a valid alias configuration
[error]               /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/fs doesn't exist
[error]             .wasm
[error]               Field 'browser' doesn't contain a valid alias configuration
[error]       looking for modules in /Users/leo/work/git/airframe/node_modules
[error]         No description file found
[error]         Field 'browser' doesn't contain a valid alias configuration
[error]         using description file: /Users/leo/work/git/airframe/node_modules/fs/package.json (relative path: .)
[error]           no extension
[error]             Field 'browser' doesn't contain a valid alias configuration
[error]               /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/fs.wasm doesn't exist
[error]             .mjs
[error]               Field 'browser' doesn't contain a valid alias configuration
[error]             /Users/leo/work/git/airframe/node_modules/fs is not a file
[error]           .wasm
[error]             Field 'browser' doesn't contain a valid alias configuration
[error]               /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/fs.mjs doesn't exist
[error]             .js
[error]               Field 'browser' doesn't contain a valid alias configuration
[error]             /Users/leo/work/git/airframe/node_modules/fs.wasm doesn't exist
[error]           .mjs
[error]             Field 'browser' doesn't contain a valid alias configuration
[error]               /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/fs.js doesn't exist
[error]             .json
[error]               Field 'browser' doesn't contain a valid alias configuration
[error]             /Users/leo/work/git/airframe/node_modules/fs.mjs doesn't exist
[error]           .js
[error]             Field 'browser' doesn't contain a valid alias configuration
[error]               /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/fs.json doesn't exist
[error]             as directory
[error]               /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/fs doesn't exist
[error]             /Users/leo/work/git/airframe/node_modules/fs.js doesn't exist
[error]           .json
[error]             Field 'browser' doesn't contain a valid alias configuration
[error]             /Users/leo/work/git/airframe/node_modules/fs.json doesn't exist
[error]           as directory
[error]             existing directory
[error]               use ./index.js from main in package.json
[error]                 using description file: /Users/leo/work/git/airframe/node_modules/fs/package.json (relative path: .)
[error]                   Field 'browser' doesn't contain a valid alias configuration
[error]                   using description file: /Users/leo/work/git/airframe/node_modules/fs/package.json (relative path: ./index.js)
[error]                     no extension
[error]                       Field 'browser' doesn't contain a valid alias configuration
[error]                       /Users/leo/work/git/airframe/node_modules/fs/index.js doesn't exist
[error]                     .wasm
[error]                       Field 'browser' doesn't contain a valid alias configuration
[error]                       /Users/leo/work/git/airframe/node_modules/fs/index.js.wasm doesn't exist
[error]                     .mjs
[error]                       Field 'browser' doesn't contain a valid alias configuration
[error]                       /Users/leo/work/git/airframe/node_modules/fs/index.js.mjs doesn't exist
[error]                     .js
[error]                       Field 'browser' doesn't contain a valid alias configuration
[error]                       /Users/leo/work/git/airframe/node_modules/fs/index.js.js doesn't exist
[error]                     .json
[error]                       Field 'browser' doesn't contain a valid alias configuration
[error]                       /Users/leo/work/git/airframe/node_modules/fs/index.js.json doesn't exist
[error]                     as directory
[error]                       /Users/leo/work/git/airframe/node_modules/fs/index.js doesn't exist
[error]               using path: /Users/leo/work/git/airframe/node_modules/fs/index
[error]                 using description file: /Users/leo/work/git/airframe/node_modules/fs/package.json (relative path: ./index)
[error]                   no extension
[error]                     Field 'browser' doesn't contain a valid alias configuration
[error]                     /Users/leo/work/git/airframe/node_modules/fs/index doesn't exist
[error]                   .wasm
[error]                     Field 'browser' doesn't contain a valid alias configuration
[error]                     /Users/leo/work/git/airframe/node_modules/fs/index.wasm doesn't exist
[error]                   .mjs
[error]                     Field 'browser' doesn't contain a valid alias configuration
[error]                     /Users/leo/work/git/airframe/node_modules/fs/index.mjs doesn't exist
[error]                   .js
[error]                     Field 'browser' doesn't contain a valid alias configuration
[error]                     /Users/leo/work/git/airframe/node_modules/fs/index.js doesn't exist
[error]                   .json
[error]                     Field 'browser' doesn't contain a valid alias configuration
[error]                     /Users/leo/work/git/airframe/node_modules/fs/index.json doesn't exist
[error] Failure on parsing the output of webpack: No content to map due to end-of-input
[error]  at [Source: java.lang.ProcessImpl$ProcessPipeInputStream@7169f3f0; line: 1, column: 0]
[error] You can try to manually execute the command
[error] node /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/node_modules/webpack/bin/webpack --bail --profile --json --mode development /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/airframe-rx-widget-test-fastopt-loader.js --output /Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/airframe-rx-widget-test-fastopt-bundle.js
[error]
[error] stack trace is suppressed; run last widgetJS / Test / jsEnvInput for the full output
[error] (widgetJS / Test / jsEnvInput) Non-zero exit code: 1
[error] Total time: 6 s, completed Feb 25, 2020, 11:16:00 PM

With requiresJsDomEnv in Test := false:

RxWidgetTest:
NodeJSTest:
2020-02-26T07:41:14.022Z  info [NodeJSTest]
hello
  - (NodeJSTest.scala:28)
 - render nested components 3.00ms << error: ReferenceError: window is not defined
 - use scala-js-node-js-v12 4.00ms

scala.scalajs.js.JavaScriptException: ReferenceError: window is not defined
  at org.scalajs.dom.package$.window$lzycompute(/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/https:/raw.githubusercontent.com/scala-js/scala-js-dom/v1.0.0/src/main/scala/org/scalajs/dom/package.scala:219:40)
  at org.scalajs.dom.package$.window(/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/https:/raw.githubusercontent.com/scala-js/scala-js-dom/v1.0.0/src/main/scala/org/scalajs/dom/package.scala:219:12)
  at org.scalajs.dom.package$.document$lzycompute(/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/https:/raw.githubusercontent.com/scala-js/scala-js-dom/v1.0.0/src/main/scala/org/scalajs/dom/package.scala:220:38)
  at org.scalajs.dom.package$.document(/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/https:/raw.githubusercontent.com/scala-js/scala-js-dom/v1.0.0/src/main/scala/org/scalajs/dom/package.scala:220:12)
  at wvlet.airframe.rx.html.DOMRenderer$.createNode(/Users/leo/work/git/airframe/airframe-rx/.js/src/main/scala/wvlet/airframe/rx/html/DOMRenderer.scala:37:35).widget.RxWidgetTest 0s
  at wvlet.airframe.rx.html.DOMRenderer$.traverse$1(/Users/leo/work/git/airframe/airframe-rx/.js/src/main/scala/wvlet/airframe/rx/html/DOMRenderer.scala:48:42)
  at wvlet.airframe.rx.widget.RxWidgetTest.render(/Users/leo/work/git/airframe/airframe-rx/.js/src/main/scala/wvlet/airframe/rx/html/DOMRenderer.scala:63:13)
  at {anonymous}()(/Users/leo/work/git/airframe/airframe-rx-widget/src/test/scala/wvlet/airframe/rx/widget/RxWidgetTest.scala:38:22)
  at scala.scalajs.runtime.AnonFunction0.apply(/Users/leo/work/git/airframe/airframe-rx-widget/.js/target/scala-2.12/scalajs-bundler/test/https:/raw.githubusercontent.com/scala-js/scala-js/v1.0.0/library/src/main/scala/scala/scalajs/runtime/AnonFunctions.scala:22:30)

[sjrd]

The jsdom environment, by definition and design, emulates a browser environment. So yes, it is going to hide Node.js-only functionality like the fs module. That is by design.

If you want something else to happen, you should override jsEnv in Test with your own subclass of JSEnv (possibly subclass of JSDOMNodeJSEnv) that does what you want.

[exoego]

I also need dom testing and node.js module at same time, so I will try to implement subclass of JSEnv.

[exoego]

Ah, this seems already discussed in https://github.com/scalacenter/scalajs-bundler/issues/197 and there was some attempts in #198 and https://github.com/scala-js/scala-js-env-jsdom-nodejs/pull/19

[xerial]

@sjrd Good to know that. Do you have any idea how we can expose node.js functionality at jsEnv even though jsdom is used? (rather, my question would be how it hides node.js functionality? )

A good use case of using node.js and jsdom together is implementing a testing framework like Snapshot Testing in JEST https://jestjs.io/docs/en/snapshot-testing. Rendering DOM elements with jsdom and saving/loading a snapshot of HTML texts to local files using node.js makes easier writing tests around DOM manipulation.

In this context, we only need to expose node.js interface for testing purpose. In production, we anyway can't use them together.

[exoego]

IIUC, node.js functionality is hidden because scala-js-env-jsdom-nodejs creates a sandbox using nodejs'svm module. https://github.com/scala-js/scala-js-env-jsdom-nodejs/blob/master/jsdom-nodejs-env/src/main/scala/org/scalajs/jsenv/jsdomnodejs/JSDOMNodeJSEnv.scala#L176

It seems that vm allows users to specify which objects are exposed to sandbox, so require can be exposed too. https://github.com/nodejs/help/issues/1613 this also may help. https://github.com/nodejs/help/issues/761