spark-rapids
spark-rapids copied to clipboard
[FEA] Improve AQE testing
Is your feature request related to a problem? Please describe. We don't have a good framework for testing AQE. For example, see https://github.com/NVIDIA/spark-rapids/pull/6250, which was difficult to test.
Describe the solution you'd like Design a framework for testing AQE.
This could be a good candidate to try out query fuzzing, By generating deeply nested joins and randomly tagging parts of the plan with willNotWorkOnGpu and the other tagging methods in RapidsMeta we will likely hit more edge cases.
Describe alternatives you've considered None
Additional context None