connectors
connectors copied to clipboard
Add 'equivalent' for type comparison
Currently, comparison expressions fail with IllegalArgumentException for decimal type arguments that don’t have equal precision/scale. This should be valid (and is in Spark SQL). This impacts IN and all the binary comparison operators.
This change is to add equivalent method, to only be used internally by our code to compare data types.
Refer https://github.com/delta-io/connectors/issues/222 for detail.
Please add a description explaining the need for this.
Also, is this how Spark compares types?
@tdas @allisonport-db thanks for review. Add some updates.
Looks good usage wise. Can you add tests for decimals of the same value with different scales? These should be equal.
Can you also make sure the tests you've added fail without the fix? I think one test is comparing decimals of the same scale I don't think this is testing anything new compared to our existing tests.
@allisonport-db I've updated few more test. Please check.
Also, could you give me some advice to fix on Failing because of negative scalastyle result issue in PR check?
@allisonport-db updated, but wonder why build has been failed...
Can you sign off your latest commit as detailed at the bottom of this page? https://github.com/delta-io/delta/blob/master/CONTRIBUTING.md
@allisonport-db updated. Is sign-off need for every PR?
Yes, I also added it to your PR description. Thanks, I'm merging this