great_expectations icon indicating copy to clipboard operation
great_expectations copied to clipboard

expect_column_values_to_not_be_null duplicates columns in unexpected values query

Open kujaska opened this issue 9 months ago • 0 comments

Describe the bug expect_column_values_to_not_be_null duplicates columns in unexpected values query:

To Reproduce if you have defined checkpoint like this: a) query as Select col1, col2, col3 FROM table1

b) runtime_configuration

runtime_configuration: { "result_format": { "result_format": "COMPLETE", "unexpected_index_column_names": ["col1", "col2", "col3" ], "return_unexpected_index_query": True, "partial_unexpected_count": 20 } }

and expectation expect_column_values_to_not_be_null for col1

then you will get this SQL in checkpoint result JSON and DataDocs:

To retrieve all unexpected values... SELECT col1, col2, col3, col1 FROM [#gx_temp_123] WHERE col1 IS NULL;

  • notice 2nd occurrence of col1

Expected behavior unexpected values query does not duplicate columns:

SELECT col1, col2, col3 FROM [#gx_temp_123] WHERE col1 IS NULL;

Environment (please complete the following information):

  • Operating System: Linux (Ubuntu)
  • Great Expectations Version: 0.18.1
  • Data Source: MS SQL
  • Cloud environment: Local virtual machine

kujaska avatar May 17 '24 10:05 kujaska