Don't create output data folder for pipelines with no output data (?)

[hardik-vala]

Some of my pipelines don't write any data to file, but a corresponding data folder is created, regardless. Do you think it makes sense to not create an output folder in such cases? Or is this a case of me not using xp right?

[druths]

I've given this some thought and I think having a directory created regardless is a helpful invariant - it ensures that the directory is there for future pipelines that might want to write to it.

If you don't want a directory created, you can set the pipeline to use file prefixing by putting the line

prefix file

in the globals section of the pipeline (see the docs for details). Nothing will be created unless you explicitly call $PLN(...).

What are your thoughts?

[hardik-vala]

Makes sense. Although, what would be the use case for future pipelines to write to the directory of a pipeline?

[druths]

Admittedly, probably you wouldn’t want downstream pipelines writing to the directory of another pipeline. I’m more thinking that processes outside of xp pipelines might expect to be able to check the directory contents after tasks are run. I agree that it’s a weak use case. I imagine that xp would be fine either way.

From: Hardik Vala [mailto:[email protected]] Sent: March 12, 2016 7:00 PM To: druths/xp [email protected] Cc: Drex [email protected] Subject: Re: [xp] Don't create output data folder for pipelines with no output data (?) (#9)

Makes sense. Although, what would be the use case for future pipelines to write to the directory of a pipeline?

— Reply to this email directly or view it on GitHub https://github.com/druths/xp/issues/9#issuecomment-195836875 . https://github.com/notifications/beacon/ABfl_cUJVjlGkPjIXE3LHjvdNMATS1cpks5ps1NugaJpZM4Hp6AO.gif