great_expectations icon indicating copy to clipboard operation
great_expectations copied to clipboard

GE create empty temporary table in Vertica

Open tamir-pl opened this issue 3 years ago • 2 comments

Hi, I created a datasource for Vertica (via SQLAlchemy) and I wanted to use the batch_request to create an interactive expectation suite. GE tries to create a temporary table on Vertica with this command:

CREATE TEMPORARY TABLE "ge_tmp_f3e31285" AS SELECT * FROM test.test_table WHERE true

the DDL command produces an empty table (temporary tables in Vertica are transaction-scoped docs )

To Reproduce Steps to reproduce the behavior:

  1. create SQLAlchemy connection to Vertica
  2. create a new suite using interactive mode
  3. you get empty results

Expected behavior I would like the temporary table to include data from the source data asset.

Environment:

  • Great Expectations Version: 0.13.26

The Vertica Documentation suggests using ON COMMIT PRESERVE ROWS example: CREATE TEMPORARY TABLE tempPreserve (a int, b int) ON COMMIT PRESERVE ROWS;

tamir-pl avatar Aug 31 '21 11:08 tamir-pl

Hi @tamir-pl - thanks for raising this issue! We won't be able to prioritize making this fix for Vertica in the immediate future. That said, we welcome contributions, and I'm happy to offer help in the form of PR review, and any guidance you may need. Would this be of interest to you?

talagluck avatar Sep 01 '21 15:09 talagluck

Hello @tamir-pl @talagluck,

I am planning to add the support to run Great Expectations on Vertica. It means at the starting point, Vertica will not be fully supported, buts will cover and resolve this issue.

I will open a PR when I nearly done, hope this help!

viplazylmht avatar Aug 07 '22 15:08 viplazylmht

Closing as resolved by #6145

austiezr avatar Mar 02 '23 21:03 austiezr