deequ icon indicating copy to clipboard operation
deequ copied to clipboard

How to use deequ with Spark SQL

Open eeasterly opened this issue 3 years ago • 1 comments

Is there a simple quick start example for the usage of deequ in Spark SQL? If anyone has some suggestions or examples, that would be greatly helpful. Alternatively, it could be a feature request, as it appears there is a flavor for pySpark.

eeasterly avatar Jun 28 '22 22:06 eeasterly

I think this is a good point.. sometimes, it is not adequate to apply "constraints" on single table but to check "business logic" through "table joins" using Spark SQL. I think this will help a lot to expand the usage of deequ..

cryptopalxyz avatar Aug 08 '22 07:08 cryptopalxyz