DataQualityDashboard icon indicating copy to clipboard operation
DataQualityDashboard copied to clipboard

Should DQD run measureValueCompleteness SQL when threshold is 100?

Open don-torok opened this issue 2 years ago • 2 comments

The measureValueCompleteness will pass when the threshold is 100. Do people look at passed checks except to say, 'look DQD passed xxx out of a possible yyy checks'. Checking every column or every row is expensive.

don-torok avatar Apr 27 '22 13:04 don-torok

We actually also look at checks that pass, but do have records violating the check. Especially on a first run of DQD where you use the default thresholds.

I would argue that we need two different kinds of Pass statuses: 'Pass without failed rows' and 'Pass with failed rows'. Similar how we are introducing different Fail statuses (>threshold, error, na)

MaximMoinat avatar Apr 28 '22 08:04 MaximMoinat

@MaximMoinat I like that a lot. I will add it to our board for v2.0

clairblacketer avatar Sep 12 '22 17:09 clairblacketer