EJ Song
EJ Song
@andrei-ionescu Sorry for the delay. but changes are not small or simple; need to spend some time to review it even with small PRs. I will try to make comments...
I meant out side of RuleUtils - normalize query plan from the beginning of apply function in FilterIndexRule & denormalize at the end of the apply function.
@andrei-ionescu 1=> yep, but we could rollback(denormalize) nested columns after applying rules except for from index relation. 2=> actually.. that's the problem. it's already complicated due to applying indexes &...
@andrei-ionescu Ok let me explain more about my suggestion: queryPlan = original query plan before transformation 1) nomalizedPlan = flatten + prefix all nested columns in queryPlan ``` * Project...
Sorry about rule"s".. I meant it's FilterIndexRule. I'd like to separate "applying index" and "handling nested columns". normalizedNestedColumns only transforms nested columns - without considering indexes. (though not sure this...
@andrei-ionescu That sounds a good alternative. It's already complicated with source provider & hybrid scan (need to decrypt to read it). Let's modulize it as much as possible.. Sorry for...
@clee704 could you review this PR & above few discussion comments & share some thoughts on it?
Question: is there any case that JoinRule v2 cannot cover but JoinRule v1 can? Could we replace v1 with v2?
Could you fix the build error? Let's merge this PR for the release. Thanks!
Is the test for spark3? Could you try the instruction in https://github.com/microsoft/hyperspace/pull/478/files?