dask-sql icon indicating copy to clipboard operation
dask-sql copied to clipboard

Published 20 hours ago • verified publisher icon dask-contrib

[DF] [ENH] Update limit/offset logic to reflect upstream DataFusion changes

Open charlesbluca opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. With #588, we have once again consolidated the logic for offsets and limits. Right now, we are computing the two separately:

  • offsets are computed using a custom map_partitions function
  • limits are computed from the OFFSET result using head

This can probably be consolidated, such that the custom map_partitions function also computes limits when necessary, with head only being used when no offset is specified.

charlesbluca avatar Jun 24 '22 18:06 charlesbluca