Unit Tests

[maboa]

I'm wondering whether we could add testing to the build process. I'm thinking specifically that it would be useful to be able to check the output of our console.log function(s), to give confidence when enhancing the functionality.

Currently investigating Mocha https://mochajs.org/. Would be super-interested in hearing what others think about the idea of testing / recommendations for testing frameworks for front-end JS.

[wbamberg]

I agree about this. I wondered if it would be possible to leverage the "// expected output" comments to test that the output of our examples is what we expect. I know this won't always work, because there are cases where "// expected output" isn't literally the output.

But usually it will I think. So we could:

• write something that extracts "// expected output" from the examples
• write a test harness that executes the examples, and compares with the expected outputs
• manually check any failures, and if they are false failures (because "// expected output" isn't the literal output in this case) discard them
• add the rest to a new expectedOutput field to that page's metadata in site.json
• wire up the test harness to this new field

[maboa]

@wbamberg interesting idea - I guess this would give us extra confidence that our expected output is correct.

[schalkneethling]

@maboa I am in the process of adding Jest[https://facebook.github.io/jest/] to the repo. Will have that in with some basic tests on Monday. Once that is in, anyone and everyone can contribute moar tests :)

[schalkneethling]

Added in https://github.com/mdn/interactive-examples/pull/317

[schalkneethling]

@maboa See here: https://github.com/mdn/interactive-examples/tree/master/tests

[ikarasz]

Quick update: Currently I have a script that fetches all the example files and extracts the expected output. After this, I passed the folders to a describe.each method and the examples will be tested with it.each. Everything works well when I run this on a subset of the interactive examples but as soon as I try to test all the examples, everything will fail with a timeout after ~8mins. It is too long definitely. I try to reduce the time needed to test all the examples but it is hard if we navigate to each test one by one.

[schalkneethling]

@ikarasz Could we break them up into separate test suites and run each suite separately in parallel? THinking out loudly, I have not actually looked at how possible this is.

[ikarasz]

@schalkneethling this is what I try to achieve by creating a describe (test suit) for each folder in the js examples. I can try a describe for each example instead, it could work. I will play with this at the weekend

[ikarasz]

Ok, I found it. My fault, I just didn't run the build before testing so of course the page couldn't be loaded. The error message was not straight forward... Now I've added a pretest script to run the build before testing, it works like a charm. Currently we have 83 failing tests I think I will fix as much as I can then I come back to bob to finish my prototype. There are some tricky test cases. For example Intl.NumberFormat uses a non-breaking space before the currency but in the src it is a simple space and they are not matching. Fortunately, this is the only one so far that cannot be fixed easily.