nipype
Add CLI to pydra
add a CLI and an option to clean a cachedir
when pydra runs, it uses lockfiles, but these stay if the process gets cancelled behind the scenes. a subsequent rerun without clearing the cache would result in pydra waiting forever for these tasks to finish. the cli clean option should remove folder associated with the lockfile and then the lockfile themselves. obviously if two users are using the same cache, this would not work cleanly. but users can always point to each others cache as readonly locations.
the CLI can be extended to introspect cache directory, messages, pickle files, etc.,.
here is example code:
from glob import glob
import os
import shutil
fl = glob('cache-wf/*.lock')
checksums = [val.split('_')[1][:64] for val in fl]
paths = []
for checksum in checksums:
paths.extend(glob(f"cache-wf/*{checksum}*"))
for path in paths:
if os.path.isdir(path):
shutil.rmtree(path)
else:
os.unlink(path)
@satra - i wanted to clarify your idea behind the cli, this is meant to be cli to clean or check the results after running the task or workflow, is that right?
