data icon indicating copy to clipboard operation
data copied to clipboard

`.fork(n)` returns original datapipe for `n==1` instead of singleton list

Open sehoffmann opened this issue 2 years ago • 0 comments

🐛 Describe the bug

pipe.fork(n) returns the original pipe when n == 1 and not a list with 1 element. This introduces two issues:

  1. It's unexpected to the user
  2. It requires the user to add extra code to handle this edgecase if the number of forks is dynamically determined during runtime

For instance, consider the following two snippets that will fail if n==1:

pipes = IterableWrapper(self.years).fork(n_vars)
for pipe in pipes:
     process_pipe(pipe)

=> iterates over elements of the original pipe and not over the actual pipes.

pipes = IterableWrapper(self.years).fork(n_vars)
for idx in range(n_vars):
     process_pipe(pipes[idx])

=> Will raise an NotImplementedError from Dataset.__getitem__().

To handle this case, the user needs to add extra code specifically for n==1.

Versions

nightly / latest

sehoffmann avatar May 17 '23 15:05 sehoffmann