data-diff icon indicating copy to clipboard operation
data-diff copied to clipboard

Remove Preql scripts; rely on database_types.py instead

Open sirupsen opened this issue 2 years ago • 5 comments

@erezsh wdyt about removing all the Preql scripts and just rely on database_types.py instead? We can add instructions on running a single test to seed your local database if you want to just run it locally.

I think this would cut down on the amount of testing code we have to maintain

sirupsen avatar Jun 30 '22 20:06 sirupsen

If someone wants to send a PR for it, they're welcome to.

erezsh avatar Jul 05 '22 08:07 erezsh

Things that it would have to solve:

  • DDL operations (mostly create & drop, but requires translating column types)
  • Resolving table names, so it uses the right schema, handling of lower/upper-case mangling, and avoiding reserved keywords (like comment in oracle)
  • Syntax variations (there are many). examples:
    • In MySQL || is "or", not "concat". In Oracle, "concat" can only accept two arguments, but || is concat.
  • (to be updated)

So far, trying to get our tests to run on all databases, only makes me think that more of them should use Preql, not less.

erezsh avatar Jul 21 '22 08:07 erezsh

I just stumbled across this while building a driver for Exasol. Is there anything new on this topic? How should one currently deal with testing new drivers for databases that are not supported by Preql?

nklsw avatar Aug 08 '22 11:08 nklsw

@nklsw Preql is not necessary to test new drivers. Read this guide: https://data-diff.readthedocs.io/en/latest/new-database-driver-guide.html

erezsh avatar Aug 08 '22 17:08 erezsh

@erezsh Do I understand you correctly that I should not include a new database in TEST_DATABASES? Just out of interest: What are these tests for and why should not all supported databases be tested there?

nklsw avatar Aug 09 '22 05:08 nklsw

Task completed.

erezsh avatar Oct 14 '22 12:10 erezsh