blaze
blaze copied to clipboard
[AURON #1745] Introduce auron-spark-tests submodule for correctness testing
Which issue does this PR close?
First PR for #1745 , establish the skeleton for correctness testing
Rationale for this change
N/A.
What changes are included in this PR?
We introduced submodule auron-spark-tests for correctness testing.
It has submodules: common and spark33.
commoncontains the helper classes for the whole correctness testing: setup the native engine enabled spark conf, enable/disable some test cases etc.spark33is the submodule that contains test cases for Spark 3.3, in this PR we included one test suite: AuronStringFunctionsSuite for demo purpose.
Each sparkxx submodule contains a AuronSparkTestSettings which controls
what test suites/test cases to enable/disable:
enableSuite[AuronStringFunctionsSuite]
// See https://github.com/apache/auron/issues/1724
.exclude("string / binary substring function")
We actually do find a bug for substring function, so I disabled corresponding
test here.
In different Spark version's module, we might have different enables/disables.
Are there any user-facing changes?
No.
How was this patch tested?
Added UTs.