optd icon indicating copy to clipboard operation
optd copied to clipboard

Substrait conversion layer

Open jurplel opened this issue 1 year ago • 2 comments

  • Put it in a separate crate, parallel to [optd-datafusion-bridge](https://github.com/cmu-db/optd/tree/main/optd-datafusion-bridge)
  • There exists a crate under the datafusion tree, datafusion-substrait, that claims to convert from datafusion to substrait representations. However, on the physical side (which is what we need to convert), it seems like it has the same problem that would be encountered if we were to send impl ExecutionPlan trait objects around, in that it doesn't seem to know the type or any information about each node. It seems like we wouldn't want to use it, in this case, and instead convert directly from optd nodes. I may be wrong on this---check for yourself here.
  • Bonus points: make sure serialization is working (can be a separate/later PR)

jurplel avatar Feb 03 '24 18:02 jurplel