optd icon indicating copy to clipboard operation
optd copied to clipboard

Decouple datafusion's logical optimization/conversion

Open jurplel opened this issue 1 year ago • 2 comments

Remove:

        let batches = df.collect().await?;

from datafusion-optd-cli/src/exec.rs

because it will internally run datafusion's logical optimizer. We should try to call the other collect function instead, after running our own optimizer and converting the optd plan into datafusion's (impl ExecutionPlan). This will allow us to run our end-to-end optimized query on datafusion.

jurplel avatar Feb 01 '24 19:02 jurplel