great_expectations
great_expectations copied to clipboard
Issue with ignore_row_if argument in certain expectations
I've encountered a problem with the ignore_row_if
argument in three expectations: expect_column_pair_values_a_to_be_greater_than_b
, expect_multicolumn_sun_to_equal
, and expect_compound_columns_to_be_unique
. While the argument works as expected when either_value_is_missing
or any_value_is_missing
is passed, it behaves unexpectedly when both_values_are_missing
/ all_values_are_missing
or neither
/ never
is used.
Specifically, when using both_values_are_missing
/ all_values_are_missing
or neither
/ never
in the ignore_row_if
argument, the following metrics are incorrect:
- unexpected_count
- unexpected_percent
- partial_unexpected_list
- unexpected_percent_total
- unexpected_percent_nonmissing
This issue impacts the accuracy of the expectations and could lead to incorrect interpretations of data quality.
Steps to Reproduce:
- Use the affected expectations (
expect_column_pair_values_a_to_be_greater_than_b
,expect_multicolumn_sun_to_equal
,expect_compound_columns_to_be_unique
). - Pass
both_values_are_missing
/all_values_are_missing
orneither
/never
in theignore_row_if
argument. - Observe the incorrect values for the mentioned metrics.
Expected Behavior:
The ignore_row_if
argument should accurately handle the specified conditions ( both_values_are_missing
/ all_values_are_missing
or neither
/ never
) and produce correct metrics for the affected expectations.
Additional Context:
- Great Expectations version: 0.18.12
- Python version: 3.9.6
- snowflake-connector-python: 3.8.1
- snowflake-sqlalchemy: 1.5.1