pypiper icon indicating copy to clipboard operation
pypiper copied to clipboard

Pipeline log file could collide with filepath given to logmuse

Open vreuter opened this issue 4 months ago • 2 comments

From consideration of #210 , it becomes apparent that a user could specify a path to a logfile for a pipeline manager's logger to write to that would collide with the path the manager will generate for its own log file. This would almost surely be inadvertent and would lead to unexpected behavior like log file overwrites, duplicate messages, and possibly a race condition on file writes.

To protect against this, I think that in any pipeline manager constructor, we should...

  1. check that there's not this collision
  2. if there is a collision, it should be an exception

It should be an exception rather than a warning because... a) It's truly exceptional b) It's almost surely unintended c) It would lead to undesirable behavior d) it would be at the start of almost any pipeline, so raising an exception wouldn't risk wasting a lot of compute progress e) it's unrecoverable -- there's not really a suitable automatic adjustment to make; the hypothetical filepaths in question (logmuse logfile and pypiper log file) may be expected to be present with particular values, e.g. for consumption by other tools, and so doing something like injecting some distinguishing identifier into the filename suffix doesn't seem suitable

@nsheff would you accept (in principle, code review TBD ;) ) a pull request to this effect?

vreuter avatar Feb 17 '24 20:02 vreuter