Return metric value and threshold for validation functions that have them.

[jeromelecoq]

The purpose of this is to :

• Be able to debug when validation is failed
• Have a record of data-trends in the QC system that goes beyond pass and failed, if available.

[dougollerenshaw]

@ryval: If a validation function were to return additional information, how should it be formatted?

[ryyesterday]

@dougollerenshaw: I don't use the direct output of the validation functions, but rather the output of the generate_qc_report function. I'd prefer that generate_qc_report continues to return an un-nested dictionary, where the values are either a bool, int, or float, if possible.

I do some minor tweaking of the output of generate_qc_report to make it mongoDB friendly, so if you are unable to keep the current structure I could enforce it here. http://stash.corp.alleninstitute.org/projects/SSCI/repos/ophysextractor/browse/ophysextractor/datasets/change_detection.py#87-102

[dougollerenshaw]

OK, so it looks like you're function doesn't care if the output is a bool, int or float. But I'm not sure how to deal with a function that should fail, but also has a value associated with it, given the current architecture of qc.py:

https://github.com/AllenInstitute/visual_behavior_analysis/blob/master/visual_behavior/validation/qc.py

For example, the metric checking dropped frames currently returns False if the fraction is above 0.01. The check_session_passes method returns all(metrics_to_validate.values()). So, if the dropped frame function was the third validation function, (True,True,False) currently returns False. But if the validation function output the actual fraction of dropped frames instead, (True,True,0.2) would return True.

Let's wait until @neuromusic is back to get his take. And rather than a wholesale change in the validation functions, I think it'd be more efficient output non-boolean values only for those functions that are occasionally (or regularly) failing.