apache
[WIP][GLUTEN-7690][CORE][CH][VL] Remove GlutenConfig from ColumnarRuleApplier
What changes were proposed in this pull request?
This PR proposes to remove GlutenConfig from ColumnarRuleApplier.
GlutenConfig prevents the dynamically modified configurations in interactive scenarios.
In fact, we should deprecate the GlutenConfig step by step.
How was this patch tested?
integration tests
Thanks for opening a pull request!
Could you open an issue for this pull request on Github Issues?
https://github.com/apache/incubator-gluten/issues
Then could you also rename commit message and pull request title in the following format?
[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}
See also:
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
GlutenConfig prevents the dynamically modified configurations in interactive scenarios.
@beliefer Would you like to post a code example of the interactive case you mentioned? Or probably to add a test case to guard against it?
GlutenConfig prevents the dynamically modified configurations in interactive scenarios.
@beliefer Would you like to post a code example of the interactive case you mentioned? Or probably to add a test case to guard against it?
Spark already have a lot of test cases. So I added the code example of the interactive case into the description of issue
@beliefer Aside of the constructor changes for GlutenConfig, I think 2 parts of code cleanups can be split and merged first. Would you want to split out the following parts from the PR?
- Rule constructor simplifications: E.g.,
RewriteDateTimestampComparisonRule(session: SparkSession, conf: SQLConf)toRewriteDateTimestampComparisonRule(conf: SQLConf)orRewriteDateTimestampComparisonRule(session: SparkSession) - Naming improvements: E.g.,
columnarConftoglutenConf
Thanks!