controlling what is checked

[kieranjmartin]

i'm noticing that we are getting to the point of having a huge number of options to turn off warnings for particular things. I'm thinking that it would actually be useful for the user to be able to specify which of any tests they want to be included.

I'm thinking something like a named list with

list(identify_class_differences = "WARNING", identify_extra_cols = "ERROR", identify_extra_rows = "QUIET")

(if we have error for a test, I'm not sure if we still keep working and error at end, or stop there).

I also theoretically like the idea of skipping a test, but i'm not sure how to handle that when some tests we do rely on the results of earlier ones (column comparison assumes that classes are the same)

[kieranjmartin]

On reflection, this is pretty covered in #26, but I guess is slightly extended.

[gowerc]

@kieranjmartin ,

Thinking about this some more I'm not sure it makes sense to add options to arbitrarily selecting which tests to run as a lot of our downstream code is dependent on assumptions made from the prior checks being true or not.

Indeed suppressing it at the warning level (during the print method) makes more sense to me. Ultimately nearly all of our checks are cheap and don't take much time to run so it doesn't matter if we still run them but merely suppress the warning message.

The only exception being the value differences which I think we should have an option to not run entirely (thus providing the "structure only check" as talked about in #26 .

From a design perspective though where should these options be stored ? I'm assuming as attributes on the diffdf object itself ?

[kieranjmartin]

Couldn't we just wipe it entirely? That is, we run the test, but just don't store the results (at the end we check options for any tests to remove and just set to null on the object)