Ahmed Hussein
Ahmed Hussein
**Describe the bug** Currently, the AutoTuner relies on the `workerInfo` to extract the system properties. If it fails to load the information, then it fails to make recommendations. **Suggestion** The...
Followup on #6301 and the ~~PR #6301~~ PR #6338 Some remaining comments in the PR that were not addressed yet: - https://github.com/NVIDIA/spark-rapids/pull/6301/files/88149ac1084f2e8ee0f8650e455d3a491204dede#r944474110 make recommendation for 'spark.sql.shuffle.partitions' based on spill size...
**Describe the bug** Projection used in `FileScanRDD` removes the partitioning columns from the produced row. The latter had different schema than expected by the consumers, and was missing part of...
This is triggered by #6165 While working on a fix for #6165, I saw some inconsistency on how the plugin handles errors. For example, I found that Cast uses inconsistent...
**Is your feature request related to a problem? Please describe.** This is a followup to render the output of the auto-tuner as HTML. **Related Context** - https://github.com/NVIDIA/spark-rapids/issues/6300
## Report needed documentation **Report needed documentation** As a followup on issue #6300 and its PR #6301, we need to update the Tools documentation to describe the new feature. **Describe...
**Is your feature request related to a problem? Please describe.** I wish the AutoTuner is integrated to the DataProc
**Is your feature request related to a problem?** I wish the RAPIDS Accelerator for Apache Spark would have a tool that recommends Spark configurations for users to run Spark on...
### Context - https://github.com/apache/spark/commit/07a5e5f53b ### What changes were proposed in SPARK pull request? Currently, DS V2 aggregate push-down cannot work with DS V2 Top N push-down (`ORDER BY col LIMIT...
### Context - Spark-3.4 Commit: https://github.com/apache/spark/commit/04ea0acf96 - Existing Rapids Issues - https://github.com/NVIDIA/spark-rapids/issues/5902 - https://github.com/NVIDIA/spark-rapids/issues/5120 ### What changes were proposed in SPARK pull request? 1. to support casting of the integral...