great_expectations icon indicating copy to clipboard operation
great_expectations copied to clipboard

I have problem with conditions

Open SarvarIslamov opened this issue 1 month ago • 1 comments

I have problem with conditions I want to check the column 'chat_id' for not being Null if and only if column 'whatsapp_lead_source' = 'openlines'

I didn't find any information about it I am using Redshift as connection (datasource)

validator.expect_column_values_to_not_be_null(
    column='chat_id',
    condition_parser='great_expectations__sqlalchemy',
    row_condition="whatsapp_lead_source = 'openlines'"
)

I get error


InvalidExpectationConfigurationError Traceback (most recent call last) Cell In[99], line 1 ----> 1 validator.expect_column_values_to_not_be_null( 2 column='chat_id', 3 condition_parser='great_expectations__sqlalchemy', 4 row_condition="whatsapp_lead_source = 'openlines'" 5 )

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/validator/validator.py:484, in Validator.validate_expectation..inst_expectation(*args, **kwargs) 478 def inst_expectation(*args: dict, **kwargs): # noqa: PLR0912 479 # this is used so that exceptions are caught appropriately when they occur in expectation config 480 481 # TODO: JPC - THIS LOGIC DOES NOT RESPECT DEFAULTS SET BY USERS IN THE VALIDATOR VS IN THE EXPECTATION 482 # DEVREL has action to develop a new plan in coordination with MarioPod --> 484 expectation_kwargs = recursively_convert_to_json_serializable(kwargs) 486 meta: Optional[dict] = expectation_kwargs.pop("meta", None) 488 basic_default_expectation_args: dict = { 489 k: v 490 for k, v in self.default_expectation_args.items() 491 if k in Validator.RUNTIME_KEYS 492 }

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/data_asset/util.py:100, in recursively_convert_to_json_serializable(test_obj) 84 def recursively_convert_to_json_serializable( 85 test_obj: dict, 86 ) -> dict: 87 """ 88 Helper function to convert an object to one that is serializable 89 (...) 98 99 """ --> 100 return _recursively_convert_to_json_serializable(test_obj)

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/data_asset/util.py:129, in _recursively_convert_to_json_serializable(test_obj) 127 for key in test_obj: 128 if key == "row_condition" and test_obj[key] is not None: --> 129 ensure_row_condition_is_correct(test_obj[key]) 130 # A pandas index can be numeric, and a dict key can be numeric, but a json key must be a string 131 new_dict[str(key)] = recursively_convert_to_json_serializable(test_obj[key])

File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/data_asset/util.py:218, in ensure_row_condition_is_correct(row_condition_string) 206 """Ensure no quote nor \\n are introduced in row_condition string. 207 208 Otherwise it may cause an issue at the reload of the expectation. (...) 215 the pandas query string 216 """ 217 if "'" in row_condition_string: --> 218 raise InvalidExpectationConfigurationError( 219 f"{row_condition_string} cannot be serialized to json. " 220 "Do not introduce simple quotes in configuration." 221 "Use double quotes instead." 222 ) 223 if "\n" in row_condition_string: 224 raise InvalidExpectationConfigurationError( 225 f"{row_condition_string!r} cannot be serialized to json. Do not introduce \n in configuration." 226 )

InvalidExpectationConfigurationError: whatsapp_lead_source = 'openlines' cannot be serialized to json. Do not introduce simple quotes in configuration.Use double quotes instead.

SarvarIslamov avatar May 25 '24 07:05 SarvarIslamov