datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Add `SessionContext`/`SessionState::create_physical_expr()` to create `PhysicalExpressions` from `Expr`s

Open alamb opened this issue 1 year ago • 0 comments

~Draft as it builds on https://github.com/apache/datafusion/pull/10331~

Which issue does this PR close?

Closes https://github.com/apache/datafusion/issues/10181

Rationale for this change

Rationale:

There important use cases for creating Exprs and executing them as a PhysicalExpr outside the context of a query (for example, to apply delete predicates in delta.rs)

At the moment this is possible but is often tricky as users have to manually invoke type coercion and simplification rules to get the Expr into a form that can be executed (see example here)

After #8045, to use certain expressions that translate to functions it is also important to to apply function rewrite rules. See https://github.com/apache/datafusion/issues/10181 for more details

Thus having a proper, tested, documented public API that does this I think is valuable and will prevent future regressions as we continue refactoring.

What changes are included in this PR?

Changes

  1. Add SessionContext::create_physical_expr() and SessionState::create_physical_expr()
  2. Apply FunctionRewrites
  3. Update examples to use the new API
  4. Add tests for the new API

Are these changes tested?

Yes, new tests

Are there any user-facing changes?

Yes: New API

alamb avatar May 01 '24 17:05 alamb