airflow-provider-great-expectations icon indicating copy to clipboard operation
airflow-provider-great-expectations copied to clipboard

Feature Request: run EXPERIMENTAL expectation (from great_expectations_experimental library) from Airflow?

Open kujaska opened this issue 1 year ago • 1 comments

Hi! Is it possible to run 1) EXPERIMENTAL expectation (from great_expectations_experimental library) from Airflow?

example: expect_queried_column_values_to_exist_in_second_table_column

Simple import to DAG does not help:

from great_expectations_experimental.expectations.expect_queried_column_values_to_exist_in_second_table_column import ExpectQueriedColumnValuesToExistInSecondTableColumn

  • after DAG run getting this text in DataDocs instead of the expectation result:

expect_queried_column_values_to_exist_in_second_table_column(**{'batch_id': '0120cd462e58ed32be35bc92c0ae', 'template_dict': {'condition': '1=1', 'first_table_column': 'PROV_ID', 'second_table_column': 'PROV_ID', 'second_table_full_name': 'LINC'}}) (edited)

2) a custom expectation from great_expectations/plugins/expectations folder? could it be run from Airflow? how? https://docs.greatexpectations.io/docs/guides/expectations/creating_custom_expectations/how_to_use_custom_expectations/ (edited)

kujaska avatar Apr 24 '23 08:04 kujaska