DataBiosphere
cwl streaming: also support nested types
Originally posted by DailyDreaming in https://github.com/DataBiosphere/toil/pull/3799#discussion_r714323831
┆Issue is synchronized with this Jira Story ┆Issue Number: TOIL-1031
➤ Adam Novak commented:
Right now we do some work on CWL tool inputs to see if the input ought to be streamable and try and stream it.
But we only do this for top-level File inputs, not for File objects nested inside other data structures.
We should change the code to be recursive over the whole input and find and stream-ify all the files in it.
➤ Adam Novak commented:
We’re not getting a lot of user demand for this, because I think people rarely actually pass files around in larger structures, and then within those people they rarely notice they aren’t getting streaming.
That being said, I think you hit this as soon as you have even an array of files.
➤ Adam Novak commented:
We would fix this faster if there was a conformance test that demanded it.