dask-sql
dask-sql copied to clipboard
[ENH] Improve `UNION` logic to drop duplicates once Datafusion logical planner is updated
Is your feature request related to a problem? Please describe.
https://github.com/apache/arrow-datafusion/issues/2573 will bring in changes to UNION logical plan where we can distinguish a UNION vs UNION ALL query. This will need updating dask-sql code to adapt for appropriate changes.