Support dynamic changing of filters

[randycoulman]

mix test supports a number of options for filtering which tests will be run: --include, --exclude, and --only.

Currently, if any of those options are provided to mix test.interactive, they will be forwarded on to mix test on each run.

It would be a nice addition to mix test.interactive to be able to dynamically change these filters in interactive mode without having to restart.

I don't use tags on my tests normally, so I'm not sure of the best way to implement such a feature. I would love feedback from those who would use such a feature. How should this work? What use cases need to be supported?

Some questions:

• Does it ever make sense to use both --only and --include/--exclude at the same time? Or are they mutually exclusive?
• It looks like sometimes --exclude should be specified before --include, but other times, --include should come before --exclude. How could mix test.interactive decide which order to use?
• How should this feature interact with the settings from ExUnit.configure? mix test.interactive would not likely be able to change those settings entirely. It would only be able to override them via the command-line options.

Thanks!

[jfpedroza]

Does it ever make sense to use both --only and --include/--exclude at the same time? Or are they mutually exclusive?

--only foo is equivalent to --exclude test --include foo, so I think it doesn't make sense to use both (but mix test allows it).

It looks like sometimes --exclude should be specified before --include, but other times, --include should come before --exclude. How could mix test.interactive decide which order to use?

From what I gather from the documentation, the other doesn't matter. It mentions that everything is included by default and so it needs to be excluded first, but that just means that you need to explicitly pass --exclude (or use exclude in ExUnit.configure).

How should this feature interact with the settings from ExUnit.configure? mix test.interactive would not likely be able to change those settings entirely. It would only be able to override them via the command-line options.

It should behave the same way as mix test, the command-line options are added to the ones in ExUnit.configure, they don't replace.