datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

[Docs] Library User Guide page for adding your own custom SQL syntax

Open alamb opened this issue 1 month ago • 2 comments

Is your feature request related to a problem or challenge?

in https://github.com/apache/datafusion/pull/17843, @geoffreyclaude added the RelationPlanner that, along with the existing ExprPlanner and TypePlanners allows Datafusion users to extend SQL in many powerful ways

https://github.com/apache/datafusion/pull/17843 has several great examples, but they may not be all that discoverable (easy for others to find)

Describe the solution you'd like

I think it would be even better to add a page to the documentation with a high level overview:

Describe alternatives you've considered

Add a page to the user guide https://datafusion.apache.org/library-user-guide with high level guidance of how to extend SQL

  1. Explain the usecase for extending SQL syntax (that different systems have different SQL needs and instead of forcing one SQL dialect, DataFusion allows you to extend your own)
  2. Enumerate the extension points and give examples
  • ExprPlanner: Allows one to intercept arbitrary expressions / operators (e.g. can plan the : operator for variant)
  • TypePlanner: Allow supporting arbitrary types
  • RelationPlanner (added in #17843) : Allow planning items in the FROM clause, such as PIVOT/UNPIVOT, TABLESAMPLE, etc

Additional context

I also suggest a blog for this content as well

  • https://github.com/apache/datafusion/issues/16756

I think there would be substantial overlap with the content of the user guide page (which is fine

alamb avatar Dec 04 '25 14:12 alamb

take

geoffreyclaude avatar Dec 10 '25 17:12 geoffreyclaude

@alamb I opened https://github.com/apache/datafusion/pull/19265 to close this issue. I kept it relatively concise, in line with the other docs. Let me know if you think it needs to be more detailed however!

geoffreyclaude avatar Dec 10 '25 19:12 geoffreyclaude

Thank you @geoffreyclaude -- I will try and review it soon

alamb avatar Dec 12 '25 17:12 alamb