great_expectations
great_expectations copied to clipboard
GE create empty temporary table in Vertica
Hi, I created a datasource for Vertica (via SQLAlchemy) and I wanted to use the batch_request to create an interactive expectation suite. GE tries to create a temporary table on Vertica with this command:
CREATE TEMPORARY TABLE "ge_tmp_f3e31285" AS SELECT * FROM test.test_table WHERE true
the DDL command produces an empty table (temporary tables in Vertica are transaction-scoped docs )
To Reproduce Steps to reproduce the behavior:
- create SQLAlchemy connection to Vertica
- create a new suite using interactive mode
- you get empty results
Expected behavior I would like the temporary table to include data from the source data asset.
Environment:
- Great Expectations Version: 0.13.26
The Vertica Documentation suggests using ON COMMIT PRESERVE ROWS
example:
CREATE TEMPORARY TABLE tempPreserve (a int, b int) ON COMMIT PRESERVE ROWS;
Hi @tamir-pl - thanks for raising this issue! We won't be able to prioritize making this fix for Vertica in the immediate future. That said, we welcome contributions, and I'm happy to offer help in the form of PR review, and any guidance you may need. Would this be of interest to you?
Hello @tamir-pl @talagluck,
I am planning to add the support to run Great Expectations on Vertica. It means at the starting point, Vertica will not be fully supported, buts will cover and resolve this issue.
I will open a PR when I nearly done, hope this help!
Closing as resolved by #6145