spark
spark copied to clipboard
[SPARK-54655] [SS] Reduce test time for StateStoreCoordinatorSuite
What changes were proposed in this pull request?
Reduce test time by doing the following:
- Replace Thread.sleep with eventually
- Maintenance thread pool timeout set to minimum of the timeout from SQLConf and 60 seconds
- Reduce shuffle partitions/iterations for extremely slow tests (e.g. "SPARK-51358: Snapshot uploads in $providerName are properly reported to the coordinator")
Why are the changes needed?
Test is very slow currently (>12 min)
Does this PR introduce any user-facing change?
No
How was this patch tested?
Testing only
Was this patch authored or co-authored using generative AI tooling?
No