datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Removed last usages of scalar_inputs, scalar_input_types and inputs2 to use arrow unary/binary for performance

Open buraksenn opened this issue 1 year ago • 3 comments

Which issue does this PR close?

Closes #12923 .

Rationale for this change

Copy-pasta'ing the issue description here: In https://github.com/apache/datafusion/pull/12881, https://github.com/apache/datafusion/pull/12890 it turned out that make_function_scalar_inputs_return_type may lead to less performant code. In https://github.com/apache/datafusion/pull/12909 it turned out that make_function_inputs2 may lead to less performant code.

That's why this PR refactores their usages.

What changes are included in this PR?

  • Remove make_function_scalar_inputs macro
  • Remove make_function_scalar_inputs_return_type
  • Remove make_function_inputs2
  • Refactor their usages throughout the app

Are these changes tested?

Existing testcases

I can add benchmarks if requested

buraksenn avatar Oct 16 '24 16:10 buraksenn

LGTM, thank you @buraksenn. I wonder how CI passed before the last commit 🤔

Thanks for reviewing the PR.

The reason was that in a few lines above:

let mut x = &args[0]; Thus using args[0] did not cause any errors but of course not a best practice afais.

buraksenn avatar Oct 18 '24 12:10 buraksenn

let mut x = &args[0]; Thus using args[0] did not cause any errors but of course not a best practice afais.

x is re-set to args[1] if log has two parameters, so using args[0] should have caused inconsistent behavior (sometimes pointing to the base of log, sometimes to the actual log value) in that commit. This might have uncovered a bug. I am planning to investigate it further.

berkaysynnada avatar Oct 18 '24 13:10 berkaysynnada

let mut x = &args[0]; Thus using args[0] did not cause any errors but of course not a best practice afais.

x is re-set to args[1] if log has two parameters, so using args[0] should have caused inconsistent behavior (sometimes pointing to the base of log, sometimes to the actual log value) in that commit. This might have uncovered a bug. I am planning to investigate it further.

I saw that now. Then probably test case miss this. I can add further tests on this

buraksenn avatar Oct 18 '24 13:10 buraksenn

Thanks again @buraksenn

berkaysynnada avatar Oct 21 '24 11:10 berkaysynnada