Andrew Lamb

https://github.com/apache/datafusion/pull/11060 is now merged 🎉

@thinkharderdev suggests it would be useful to be able to serialize constant parameters into the user-defined scalar function themselves rather than pass them in as expressions. So for instance if...

One way to achieve this might be a [PhysicalOptimizerRule](https://docs.rs/datafusion/latest/datafusion/physical_optimizer/optimizer/trait.PhysicalOptimizerRule.html#) that replaces relevant instances of [ScalarFunctionExpr](https://docs.rs/datafusion/latest/datafusion/physical_expr/struct.ScalarFunctionExpr.html#). However this is likely somewhat awkward to write and is not clear that these expressions...

@2010YOUY01 noted that in order to use the same APIs for built in functions and ScalarUDFs we will need to have some way to handle: > Figure out the interface...

I think the trick here will be to ensure we can still serialize such "precompiled" functions What I was thinking was maybe we can make a new `PhysicalExpr` that is...

> That seems a bit heavy handed? Why not simply augment the scalar function expr node with an interior mutable "cache" cell, that is invisible to serialization? The cell can...

> > That seems a bit heavy handed? Why not simply augment the scalar function expr node with an interior mutable "cache" cell, that is invisible to serialization? The cell...

BTW my hope it to prototype how this would work (as a ScalarUDF) by building on top of https://github.com/apache/arrow-datafusion/pull/8578 It would be pretty rad

BTW now that @jayzhan211 and I have implemented `ScalarUDF::simplify` in https://github.com/apache/arrow-datafusion/pull/9298 and we have ported the regular_expression functions to use `ScalarUDF`, I think we could actually use that API to...

FYI there is another discussion about this here: https://github.com/apache/datafusion/issues/11146