patronus icon indicating copy to clipboard operation
patronus copied to clipboard

Combos vs provided params

Open mattapperson opened this issue 9 years ago • 6 comments

Currently Patronus is looking for all potential combos of params, but then only using combos provided in the routes plugin object. This needs to change to this (recommended) flow:

  1. Validate provided test param combos (and assert that they are correct, fail if they are not)
  2. Generate the combos and check that the provided params cover all use cases (again, should be asserted)
  3. https://github.com/liftsecurity/bizzarolout is going to be split out by @jlamendo to a module we can use its ability to fuzz routes... generate lots of tests based on this too

right now we just return an array of tests... I think we should rather return an object with 3 params:

var tests = {
    user: [], // an array of tests based on user provided params
    coverage: [], // some tests/assertions, not really sure to pass/fail if the user tests have correct coverage, and if not point out what is needed
    security: [] // tests based off of bizzarolout
};

mattapperson avatar Mar 20 '15 22:03 mattapperson

@mattapperson I've made some changes and cleaned up the sorrow library, added a readme, etc. It's now publicly accessible at https://github.com/liftsecurity/sorrow

jlamendo avatar Mar 21 '15 03:03 jlamendo

First move was made here, broke out user var testing in the API. Now to add in the other 2 test types

mattapperson avatar Mar 28 '15 02:03 mattapperson

OK, so now points one and 2 are both covered in master. On to the fuzzing just as soon as https://github.com/liftsecurity/sorrow/issues/1 is fixed

mattapperson avatar Apr 07 '15 02:04 mattapperson

@mattapperson I'll be working on that this weekend :) Unfortunately I haven't had much time to work on it this week. I did add a new unicode fuzzer, however, as I needed it for some dayjob work.

jlamendo avatar Apr 07 '15 18:04 jlamendo

@jlamendo no rush, the safety of the world hangs in the balance, but take your time :P

mattapperson avatar Apr 07 '15 19:04 mattapperson

Fixed! Let me know if there's any improvements to the usability of the new features. One of the major goals of sorrow is to be easily accessible to developers, so if you think of any ways it could be improved I'm all ears.

jlamendo avatar Apr 20 '15 09:04 jlamendo