blaze
blaze copied to clipboard
Enhance Auron's Correctness Testing
Auron is a great Native Engine. But we also see some correctness related issues recently:
- https://github.com/apache/auron/issues/1674
- https://github.com/apache/auron/issues/1716
- https://github.com/apache/auron/issues/1680
Apache Gluten(https://github.com/apache/incubator-gluten) faces the similar issue, Gluten has built a great mechanism that runs all the Unit Test bundled with Spark(https://github.com/apache/incubator-gluten/tree/main/gluten-ut), but with native engine enabled. Doing this Gluten can check whether Gluten produces the same result as Vanilla Spark.
I think Auron can copy Gluten's idea to do the similar thing, to enhance the user's confidence to use Auron in production environment.