datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Implement `Eq`, `PartialEq`, `Hash` for `dyn PhysicalExpr`

Open peter-toth opened this issue 1 year ago • 1 comments

Which issue does this PR close?

Part of https://github.com/apache/datafusion/issues/12599.

Rationale for this change

Common subexpression elimination will require Arc<dyn PhysicalExpr> tree nodes to implement Eq, PartialEq, Hash.

What changes are included in this PR?

This PR:

  • Refactors PhysicalExpr trait and moves dyn_hash() to a new DynHash trait.
  • Adds blanket implementation of DynHash for types that implement Hash. This enables us to drop most of the previous dyn_hash() boilerplate implementations.
  • Introduces DynEq trait with dyn_eq().
  • Adds blanket implementation of DynEq for types that implement Eq.

Please note that:

  • This PR contains an API breaking change that dyn_hash() is moved from PhysicalExpr to separate DynHash trait.
  • This PR contains a small hack that some PhysicalExpr implementations (like BinaryExpr) got a new generic parameter called DynPhysicalExpr (e.g. struct BinaryExpr<DynPhysicalExpr: ?Sized = dyn PhysicalExpr>). This was needed due to a rust bug: https://github.com/rust-lang/rust/issues/78808. This hack: https://github.com/rust-lang/rust/issues/78808#issuecomment-1664416547 allows us to derive PartialEq instead of implementing it separately for the affected types.

Are these changes tested?

Yes, with existing tests.

Are there any user-facing changes?

No.

peter-toth avatar Oct 18 '24 13:10 peter-toth

cc @alamb

peter-toth avatar Oct 18 '24 13:10 peter-toth

On my list for review tomorrow

alamb avatar Oct 21 '24 20:10 alamb

What I would like to propose for this PR is wait until we have made the next release: #12470 and then we can merge it after that

I am thinking of how we can spread out the API upgrade pain

alamb avatar Oct 31 '24 18:10 alamb

The 43.0.0 release candidate has been made and we have started voting on it. I merged this branch up from main locally to ensure everything still compiles. It looks good so this is ready to go 🏁

THanks again @peter-toth and @berkaysynnada

alamb avatar Nov 05 '24 18:11 alamb

Thanks for review @alamb and @berkaysynnada.

peter-toth avatar Nov 05 '24 18:11 peter-toth