ShreyaGoddu

Spark Comparison fails to detect similarities in unordered data

2

Found a few cases where datacompy returns mismatches with Spark dataframe comparisons when the data is not sorted (using v0.11.3) Cases where it reports mismatches when it shouldn't: 1. Column...

Add support to compare.matches() to accept optional threshold

7

We have a number of use cases where 100% row match isn't required and 90% or another configurable value is permissible. Can there be support added to accept a custom...