snakebids
snakebids copied to clipboard
`export_sourcedir` plugin
This proposal will supersede #180 .
The plugin will allow the copying or symlinking of directories in the app to the output dir. This will allow the snakemake app to access contained files via relative files, solving the provenance issues caused by reinstalling the app in a different location
def export_sources(
paths: str | Path | list[str | Path],
*,
root: str | Path = ".",
timestamp: bool = True,
checksum: bool = False,
version: str | tuple[str, str, str] | packaging.version.Version | None = None,
symlink: bool = False,
):
"""Recursively copy given directories and files into the output dir.
Allows workflow to access files in the directories via relative paths. All exported
paths must be subdirectories of the root, which defaults to the snakemake_dir given
to SnakeBidsApp. Paths will be normalized relative to the root, then exported as-is.
If exported files are already in the output dir, the export will only be performed
on outdated files. By default, outdated files are determined by comparing the mtime
of the previously exported file to the file in the app. If a ``version``
specifier is used, checks for outdated files will only be made when the version
changes.
Parameters
==========
paths :
Paths of directories and files to export. Directories are exported recursively.
root :
Root of exported directories. All paths will be exported as relative to this
root.
timestamp :
Allow the default behaviour of updating exports whenever the exported copy is
older than the file to be exported. Defaults to ``True``.
checksum :
Find outdated exports by computing the file checksum. This is the most
robust solution to deal with snakemake's automatic provenance, but may be slow
for large files. If set, this takes precendence over ``timestamp``.
version :
A version string, tuple, or ``Version`` object to be recorded with the output
data. Outdated exports will be updated only when the version changes.
symlink :
If ``True``, symlinks will be made to exported files, rather than copying. This
method is faster when large files are being exported, but may not be compatible
with some distributed modes of snakemake.
"""
I think this feature would be quite useful, but not a priority before 1.0.
See also #109, use of a plugin like this would probably supersede any reference to those variables