Add tests to configurations

[ryan-roemer]

Something like:

test/
  fixtures/
    # Example code should _pass_ `configurations/formidable/NAME`
    pass/
      es5-browser.js
      es5-node.js
      es5-test.js
      es5.js
      es6-browser.js
      es6-node.js
      es6-react-test.js
      es6-react.js
      es6-test.js
      es6.js
    # Example code should _fail_ `configurations/formidable/NAME`
    fail/
      es5-browser.js
      es5-node.js
      es5-test.js
      es5.js
      es6-browser.js
      es6-node.js
      es6-react-test.js
      es6-react.js
      es6-test.js
      es6.js
  spec/
    pass.spec.js
    fail.spec.js

The pass|fail.spec.js scripts just read in the files and exec eslint with the respective config and file name.

Current challenge: The configs look like this:

"use strict";

module.exports = {
  "extends": "defaults/configurations/formidable/es5",
  "env": {
    "browser": true
  }
};

but `"defaults/*" isn't available. It really needs to be something like:

"use strict";

module.exports = {
  "extends": "/PATH/TO/configurations/formidable/es5",
  "env": {
    "browser": true
  }
};

Options:

• [ ] Hack require (used under the hood by eslint) in the exec'ed process (hard and hacky).
• [ ] Call eslint programmatically and use something like mock-fs to mock and mutate the files in memory.

[baer]

This is just testing glue and there would be a lot of work to do so. Is this worth the effort? The CI runs lint over the project as it is which will test that the config is properly consumable. This could probably be made more exhaustive but this level of testing might be overkill for what amounts to a set of JSON files.

[kylecesmat]

@baer maybe a good solution to start is to make sure all configurations are consumable? Right now CI is only consuming specific rules & not the configurations themselves (ie #24)

[baer]

Making sure all config is consumable is one thing but hacking require and building an exhaustive collection of fixtures is another entirely. I think before a test plan is put together there should be a few goals. Making sure tests are runnable seems like a good one.