datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard
verified publisher icon apache

[EPIC] Support converting Exprs and LogicalPlans --> SQL Strings

Open alamb opened this issue 1 year ago • 4 comments

This ticket tracks various tickets related to converting Exprs and LogicalPlans to/from SQL strings

  • [ ] https://github.com/apache/arrow-datafusion/issues/8736
  • [ ] https://github.com/apache/arrow-datafusion/issues/8661
  • [x] https://github.com/apache/arrow-datafusion/issues/9726
  • [x] https://github.com/apache/arrow-datafusion/issues/9495
  • [x] #10524
  • [x] https://github.com/apache/datafusion/issues/10550
  • [x] https://github.com/apache/datafusion/issues/10557
  • [ ] https://github.com/apache/datafusion/issues/10633

alamb avatar Mar 07 '24 21:03 alamb

I think that we now have a solid enough foundation on these tasks that it would be a good first issue for a new contributor (#9495 especially). An individual or small number of logical Exprs could be handled in a stand alone PR with a test or two. I think this project is a great way to get more familiar with DataFusion's core data structures and optimization algorithms without already having expertise in database internals.

devinjdangelo avatar Mar 19 '24 12:03 devinjdangelo

@devinjdangelo do you think we should file explicit follow on tickets, or do you think the existing PRs / documentation is clear enough about the ask (for someone who has no experience with DataFusion)?

I can file tickets if you prefer

alamb avatar Mar 19 '24 17:03 alamb

@alamb I don't think we should file a ticket for every remaining Expr variant, but we could edit the description of #9495 to list the remaining unimplemented variants and make clear that we are looking for help for anyone interested to implement even just 1 at a time in a PR + write a test.

Here is the remaining list:

  • [ ] InList
  • [ ] ScalarFunction
  • [ ] Between
  • [ ] Case
  • [ ] WindowFunction
  • [ ] Like
  • [ ] ScalarVariable
  • [ ] SimilarTo
  • [ ] Not
  • [ ] IsNotNull
  • [ ] IsTrue
  • [ ] IsFalse
  • [ ] IsUnknown
  • [ ] IsNotTrue
  • [ ] IsNotUnknown
  • [ ] Negative
  • [ ] GetIndexedField
  • [ ] TryCast
  • [ ] Sort
  • [ ] Exists
  • [ ] Wildcard
  • [ ] GroupingSet
  • [ ] Placeholder
  • [ ] OuterReferenceColumn
  • [ ] Unnest

devinjdangelo avatar Mar 19 '24 21:03 devinjdangelo

@alamb I don't think we should file a ticket for every remaining Expr variant, but we could edit the description of #9495 to list the remaining unimplemented variants and make clear that we are looking for help for anyone interested to implement even just 1 at a time in a PR + write a test.

Thank yoU @devinjdangelo -- I filed https://github.com/apache/arrow-datafusion/issues/9726

alamb avatar Mar 21 '24 17:03 alamb

Let's claim this is complete 🎉 All the existing subtasks are closed and now we are working on this in our ongoing operations

alamb avatar Nov 22 '24 14:11 alamb