legend-engine icon indicating copy to clipboard operation
legend-engine copied to clipboard

Published 20 hours ago • verified publisher icon finos

Improvements to the Rel transpiler

Open relationalai-opensource opened this issue 11 months ago • 3 comments

Various improvements to the Rel transpiler:

  • Add support for the restrict operator, ensuring that a restrict following join or project does not materialize unnecessary columns in intermediate relations in the transpiled Rel query.
  • Improvements to the way that the schema is constructed. These changes should make it easier to ensure that the required classes are added to the schema before the query can be transpiled to Rel. These changes will make it easier to transpile queries which use match.
  • Improved formatting of the output Rel query for better readability.
  • Improved debugging and logging in the case of errors during query compilation.
  • Added test cases.
  • Property mappings are now correctly used by the transpiler to map onto Rel relations corresponding to properties, whereas previously only class mappings were used.
  • Fix bug in handling of groupby.
  • General improvements to the generated Rel, for example, improved handling of isEmpty().
  • Miscellaneous code cleanup and bug fixes.

This PR builds on the work of https://github.com/finos/legend-engine/pull/2622, and therefore that PR must be merged first.

What type of PR is this?

Improvement

What does this PR do / why is it needed ?

Support for more types of query by the Rel backend. This PR also improves the logging of errors when a Pure query is not supported by the Rel transpiler.

Does this PR introduce a user-facing change?

This PR is backwards compatible in the sense that existing queries will continue to be supported without any changes.

Some function names related to schema creation have changed. See test cases for examples.

relationalai-opensource avatar Mar 13 '24 23:03 relationalai-opensource

Test Results

     748 files  ±0       748 suites  ±0   1h 5m 48s :stopwatch: +23s 12 287 tests ±0  12 122 :heavy_check_mark: ±0  165 :zzz: ±0  0 :x: ±0  15 321 runs  ±0  15 146 :heavy_check_mark: ±0  175 :zzz: ±0  0 :x: ±0 

Results for commit 9ecef312. ± Comparison against base commit 44a7171a.

github-actions[bot] avatar Mar 14 '24 01:03 github-actions[bot]

This PR can be closed because it is superseded by PR#2702 which has now been merged.

relationalai-opensource avatar Apr 10 '24 16:04 relationalai-opensource

This PR can be closed because it is superseded by https://github.com/finos/legend-engine/pull/2702. Apologies for the noise.

andrew-bate-rai avatar Apr 11 '24 16:04 andrew-bate-rai

This PR is stale because it has been open for 30 days with no activity. Please remove stale label or add any comment to keep this open. Otherwise this will be closed in 5 days.

finos-admin avatar May 12 '24 12:05 finos-admin

This PR was closed because it has been inactive for 35 days. Please re-open if this PR is still relevant.

finos-admin avatar May 17 '24 12:05 finos-admin