rx icon indicating copy to clipboard operation
rx copied to clipboard

Tests

Open cloudhead opened this issue 6 years ago • 3 comments

  • [ ] parser.rs: the parser should be property tested with something like quickcheck
  • [ ] cmd.rs: CommandLine should be tested with ascii and utf8
  • [ ] resources.rs: Gif export should be tested
  • [ ] session.rs: inputs (mouse, keyboard) should be fuzz-tested to ensure nothing can crash the program.
  • [ ] session.rs: selection/* commands should be tested in headless mode
  • [ ] session.rs: all commands that take user input should be tested with unexpected inputs to ensure there is no division by zero for example
  • [ ] lib.rs: rx's entry point should be tested with different inputs
  • [ ] Test that everything that modifies a view sets the [modified] flag on it

cloudhead avatar Jan 15 '20 21:01 cloudhead

@pop if anything above looks interesting, let me know and I can provide more detail. Would be great to have some help in making rx more robust.

cloudhead avatar Jan 15 '20 21:01 cloudhead

If I had to pick one, fuzz-testing the commands that take user input seems interesting. i.e.,

session.rs: all commands that take user input should be tested with unexpected inputs to ensure there is no division by zero for example

That said I have not dug too far into any of the topics so if there is something else you'd like to pass off to a contributor I'm open to one of the other topics.

I'm not entirely sure how to do headless testing but I'm sure once I get started writing the headless fuzz-tests would be pretty straight forward.

pop avatar Jan 18 '20 20:01 pop

Great, I also think the fuzz testing will be the biggest ROI at this point, but it's also possibly more work than the others.

A good place to start looking is the execution module and the current tests in tests/, as there's already the ability to run rx in headless mode and just feed it inputs from a script. There's two levels at which the fuzz testing could be useful:

  1. Create random Command objects with random parameters and feed them into the Session and try to make it crash.
  2. Send random keyboard/mouse inputs directly to the session.

Both might be useful, but 2. is less likely to trigger complex behavior, since most inputs won't be valid commands.

Another thing that might be helpful alongside this is to add another input which is the current session mode, so run the fuzz tests in Visual, Normal and Command mode; I think this will increase coverage quite a bit.

Finally there's the question of whether to turn the renderer and platform layer off or not. I think to start out it would make sense, ie. to use a dummy renderer (trait Renderer) and context manager (platform). This way the fuzz tests can run 100x faster and test many more inputs.

These are my initial thoughts.

cloudhead avatar Jan 19 '20 13:01 cloudhead