luigi icon indicating copy to clipboard operation
luigi copied to clipboard

Deduplicate outputs in luigi.task.flatten_output (#3106)

Open starhel opened this issue 3 years ago • 0 comments

Description

I've changed luigi.task.flatten_output implementation to deduplicate outputs from tasks which are required by more than one wrapper task. Implementation is slightly changed (from DFS to BFS), but there's no guaranteed order in docs.

Motivation and Context

Fixes #3106

Have you tested this? If so, how?

I've added unit tests for standard use cases and diamond problem.

starhel avatar Jun 10 '22 15:06 starhel