coral icon indicating copy to clipboard operation
coral copied to clipboard

Implement CoralRelNodeToCoralSqlNodeTransformer for lateral views

Open aastha25 opened this issue 2 years ago • 3 comments

Adding a transformer class to convert the intermediate Coral RelNode representation to Coral SqlNode representation for lateral views.

Ideally, this transformer should be in the coral-common module. However, the transformer currently depends on certain classes which are present in the coral-hive module. I'll move all the relevant classes to the coral-common module in the future as part of a larger refactoring effort.

Testing done: Added unit tests

aastha25 avatar Jun 03 '22 00:06 aastha25

Could we run the integration test along with the tool to show the diff between the syntax before and after the change?

wmoustafa avatar Jun 03 '22 21:06 wmoustafa

Thanks @aastha25 for the PR. I think we need to connect this to existing translations too (replacing some of their functionalities) to make sure the translations work end to end.

wmoustafa avatar Jun 05 '22 08:06 wmoustafa

@wmoustafa @ljfgem , I integrated my PR with the existing end-to-end translation pipeline for Spark. I tested it out with all unit tests and all the production views. No regression was observed.

aastha25 avatar Jul 13 '22 17:07 aastha25