data
data copied to clipboard
[WIP] added islice + flatten for iterable datapipes
Please read through our contribution guide prior to creating your pull request.
- Note that there is a section on requirements related to adding a new DataPipe.
Fixes #656
Changes
- added new iterdatapipe called islice for selecting a slice of an iterable item returned (ex: list, tuple, dict)
- added indexing as part of this islice functionality, allowing a list of indexes (or keys) to also be used in place of a slice for selection
- also added flatten iterdatapipe to allow for an item in an iterable to be flattened one level if an index or list of indices are provided
- added default for flatten (no args) that yields all items for all iterables in the provided datapipe chained together