data icon indicating copy to clipboard operation
data copied to clipboard

[WIP] added islice + flatten for iterable datapipes

Open dbish opened this issue 2 years ago • 0 comments

Please read through our contribution guide prior to creating your pull request.

  • Note that there is a section on requirements related to adding a new DataPipe.

Fixes #656

Changes

  • added new iterdatapipe called islice for selecting a slice of an iterable item returned (ex: list, tuple, dict)
  • added indexing as part of this islice functionality, allowing a list of indexes (or keys) to also be used in place of a slice for selection
  • also added flatten iterdatapipe to allow for an item in an iterable to be flattened one level if an index or list of indices are provided
  • added default for flatten (no args) that yields all items for all iterables in the provided datapipe chained together

dbish avatar Aug 13 '22 01:08 dbish