iceberg
iceberg copied to clipboard
Predicate Compiler for Evaluators
Predicates are currently created as ASTs, but are not compiled or optimized. Instead, evaluating them means traversing the tree in its entirety, something that, in a tight loop (like going over partitioning functions for all files in a table), is suboptimal.
Compiling the expressions into a minimal syntax tree that embeds the right-hand side values would let us optimize it ahead of time in several ways, including:
- Compressing
equals-or-equalstrees toin. - Dropping always-
trueand always-falseexpressions entirely.
This issue therefore includes three things:
- Representing expressions as a value-embedded AST.
- Creating an optimizer framework.
- Slowly building more and more optimizations.
This could probably be done using existing tools, without reinventing the wheel :)