spark-rapids icon indicating copy to clipboard operation
spark-rapids copied to clipboard

[FEA] Improve AQE testing

Open andygrove opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. We don't have a good framework for testing AQE. For example, see https://github.com/NVIDIA/spark-rapids/pull/6250, which was difficult to test.

Describe the solution you'd like Design a framework for testing AQE.

This could be a good candidate to try out query fuzzing, By generating deeply nested joins and randomly tagging parts of the plan with willNotWorkOnGpu and the other tagging methods in RapidsMeta we will likely hit more edge cases.

Describe alternatives you've considered None

Additional context None

andygrove avatar Aug 08 '22 13:08 andygrove