mithril.js icon indicating copy to clipboard operation
mithril.js copied to clipboard

Our tests should be running in real browsers as well as Node

Open dead-claudia opened this issue 5 years ago • 0 comments

We need to start testing our code in real browsers and in ES5 environments to avoid issues like this, reported on Gitter. It'd also make it much easier to detect and test for IE bugs, if we could get on either BrowserStack's or Sauce Labs' open source plans, but we first need to get our end together so we can even use it.

Here's the state of everything:

  • All our browser tests have a web page with all the relevant tests loaded via <script src="..."></script>.
  • Our browser test pages rely on a require pseudo-polyfill that doesn't even wrap modules in a closure when executing them.
  • Our tests exclusively use DOM mocks, missing 100% of browser-specific bugs.
  • We're only actively testing it in the latest Node, using our own mocks instead of something like JSDOM.

This is low priority for now, but I am working on it partially in my local Rollup migration branch (slated for a future PR). Here's my plan, some of which I've already got resolved locally:

  • Migrate the tests to grab their mocks from a global instance, created from a test bundle rather than the individual files.
  • Migrate the web pages to load only the tests and the generated test bundle.
  • Make ospec on npm test load the generated test bundle before executing tests in Node.
  • Create a karma-ospec to tell Karma how to handle ospec results
    • I'll need to add a feature for async test reporting, so Karma doesn't go ballistic over not receiving a single event in the ~3 seconds it takes to run everything, in case the browser briefly disconnects in the middle of test execution. (Karma's timeout is 2000ms = 2 seconds, which is a problem.)
  • Switch our browser test runner to Karma
  • Set up Travis to run it in Chrome, Firefox, and Safari using their respective launchers.
    • This is probably going to be the easiest part of this.
  • Migrate the DOM tests off of the mocks as much as practically possible.
    • Testing for ev.preventDefault() is as simple as checking ev.defaultPrevented
    • Testing for ev.stopPropagation() is as simple as firing an event that can propagate and ensuring it doesn't get found by the parent.
    • The valueSetter and typeSetter spies could just be replaced with an interim hack for elem.value and input.type, which we'd remove once we incorporate IE into the test matrix. (We can do this on Travis with their early stage Windows CI support, but note that Edge is not available without something like Sauce Labs or BrowserStack.)
  • Set up Travis to run tests in IE via karma-ie-launcher. This includes the promise polyfill, streams, and everything.
  • And then, we can then confidently say things actually work and aren't a house of cards ready to fall over and collapse.

As for the above, I only locally have the first done completely and the second done for /api/tests/*.js only.

dead-claudia avatar Jan 17 '19 14:01 dead-claudia