optd
optd copied to clipboard
cmu-db
CMU-DB's Cascades optimizer framework
Resources: - [RelMdLowerBoundCost](https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/metadata/RelMdLowerBoundCost.java) - [TopDownRuleDriver.java](https://github.com/apache/calcite/blob/b0034c9ed885644e22edc92e8ab939128ee6b615/core/src/main/java/org/apache/calcite/plan/volcano/TopDownRuleDriver.java#L618) - Related: PR https://github.com/apache/calcite/pull/1991
# Context https://github.com/cmu-db/optd/pull/88 # Current Design When the generated expr by heuristic rule already exist in another group, we merge two groups and mark the old expr as a dead...
**List of logical->logical rule files to port [from datafusion](https://github.com/apache/arrow-datafusion/blob/c43c5f14e42cbf629e079c96ce0486fc8e63d765/datafusion/optimizer/src/lib.rs)**: - [ ] "analyzer rules" (what are these? there are 6 files) - [ ] simplify_expressions (big) - [ ] common_subexpr_eliminate...
# TODO - [x] add names and is_nullable to schema - https://github.com/cmu-db/optd/pull/61 - [x] make schema types align with datafusion::arrow::Schema type (one to one mapping) - https://github.com/cmu-db/optd/pull/57 - [x] adjust...
The logical properties of the logical optimizer includes: - [ ] schema - https://github.com/cmu-db/optd/issues/56 - [ ] outer cols (columns are not defined in the underlying expr trees, eg: agg...
- Put it in a separate crate, parallel to `[optd-datafusion-bridge](https://github.com/cmu-db/optd/tree/main/optd-datafusion-bridge)` - There exists a crate under the `datafusion` tree, `datafusion-substrait`, that claims to convert from `datafusion` to `substrait` representations. However,...
Remove: ```rs let batches = df.collect().await?; ``` from `datafusion-optd-cli/src/exec.rs` because it will internally run `datafusion`'s logical optimizer. We should try to call the *other* `collect` function instead, after running our...
we can (1) collect all table scans from the plan node, (2) in the async context, retrieve the schema, and (3) remove async from schema inference.
