pinot
pinot copied to clipboard
[multistage] support additional SQL feature tracker
Many SQL features are not yet supported in multistage engine, because one of the two followings:
- stage plan node in pinot-query-planner hasn't been implemented yet.
- operator in pinot-query-runtime hasn't been implemented yet
Some of the existing SQL features missing that prevents us from fully enabling the IntegrationTestSet
- [x] enable partial hard-coded test set: https://github.com/apache/pinot/pull/9412
- [ ] LIKE
- [ ] CAST with type alias like
LONGforBIGINT - [ ] MV column support
- [x] enable partial generated test set (https://github.com/apache/pinot/pull/9452)
Some already supported by Pinot currently but not on the new engine.
- [x] IN filter clause (https://github.com/apache/pinot/issues/9345) (https://github.com/apache/pinot/pull/9374/)
- [x] RANGE filter clause (https://github.com/apache/pinot/pull/9445)
- [x] ORDER BY (https://github.com/apache/pinot/pull/9279)
- [x] HAVING (https://github.com/apache/pinot/pull/9274)
- [x] DISTINCT (https://github.com/apache/pinot/pull/9383)
- [x] CAST (https://github.com/apache/pinot/pull/9384)
- [ ] DISTINCT Aggregation
- [x] LIMIT
<start>,<count>(https://github.com/apache/pinot/pull/9496) - [ ] UNION/INTERSECT
- [ ] AGG with FILTER
Some of the features are removing limitations of current engine
- [x] support function invoker for non-standard SQL keywords like DATE_TRUNC (requires rewrite of FunctionRegistry to record not just
# of argumentsbut aDataType[]as function signature, also might as well use function catalog in calcite) (https://github.com/apache/pinot/pull/9375) - [x] support inequality join condition as post-join filter (https://github.com/apache/pinot/pull/9328)
- [x] count(*) / count(col) / sum(1) are equivalent (without null support), but the rewrite rules are not clear (https://github.com/apache/pinot/pull/9331)
- [x] support more complex join condition expressions, like
- [x]
a JOIN b USING(col)(https://github.com/apache/pinot/pull/9373) - [x]
a JOIN b ON b.col = a.col(ordering matters as currently the ref indexing is assuming left then right) (https://github.com/apache/pinot/pull/9350)
- [x]
- [x] support LEFT JOIN (https://github.com/apache/pinot/pull/9446)
- [x] support NULL value (https://github.com/apache/pinot/pull/9427)
- [ ] support NULL-safe operators
- [x] support small RIGHT table with inequality conditions. (https://github.com/apache/pinot/pull/9448)
- [x] support SEMI JOIN https://github.com/apache/pinot/pull/9367
Some of the additional items to support but not yet
- [ ] UNNEST (related https://github.com/apache/pinot/issues/7881)
- [ ] LATERAL
- [ ] OVER aggregation (related: https://github.com/apache/pinot/issues/7213)
- [ ] MATCH_REGOGNIZE
a Join b Using(col) is already supported. I'll just send out a test.