tools
tools copied to clipboard
nf-core
First attempt at writing a new command to generate an RO Crate
New command nf-core rocrate:
$ nf-core rocrate --help
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 1.10.dev0
Usage: nf-core rocrate [OPTIONS] <pipeline directory>
Make an Research Object Crate
Options:
-j, --json TEXT Path to save RO Crate metadata json
-z, --zip TEXT Path to save RO Crate zip
--help Show this message and exit.
Generates an RO Crate .zip archive of the whole workflow + metadata files, or saves just the JSON metadata file.
For example:
$ nf-core rocrate nf-core-testpipeline/ --json ro-crate-metadata.json
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 1.10.dev0
INFO Adding 38 workflow files ro_crate.py:112
INFO Saving metadata file 'ro-crate-metadata.json'
ro-crate-metadata.json
{
"@context": "https://w3id.org/ro/crate/1.0/context",
"@graph": [
{
"@id": "ro-crate-metadata.jsonld",
"@type": "CreativeWork",
"about": {
"@id": "./"
},
"conformsTo": {
"@id": "https://w3id.org/ro/crate/1.0"
}
},
{
"@id": "ro-crate-preview.html",
"@type": "CreativeWork",
"about": {
"@id": "./"
}
},
{
"@id": "./",
"@type": "CreativeWork",
"about": {
"@id": "./"
},
"datePublished": "2020-07-30 11:16:47.918409",
"hasPart": [
{
"@id": "nextflow.config"
},
{
"@id": "CODE_OF_CONDUCT.md"
},
{
"@id": "LICENSE"
},
{
"@id": "environment.yml"
},
{
"@id": "CHANGELOG.md"
},
{
"@id": "Dockerfile"
},
{
"@id": "README.md"
},
{
"@id": "gitignore"
},
{
"@id": "gitattributes"
},
{
"@id": "nextflow_schema.json"
},
{
"@id": "main.nf"
},
{
"@id": "bin/markdown_to_html.py"
},
{
"@id": "bin/scrape_software_versions.py"
},
{
"@id": "docs/usage.md"
},
{
"@id": "docs/output.md"
},
{
"@id": "docs/README.md"
},
{
"@id": "docs/images/nf-core-testpipeline_logo.png"
},
{
"@id": "github/PULL_REQUEST_TEMPLATE.md"
},
{
"@id": "github/.dockstore.yml"
},
{
"@id": "github/CONTRIBUTING.md"
},
{
"@id": "github/markdownlint.yml"
},
{
"@id": "github/workflows/branch.yml"
},
{
"@id": "github/workflows/push_dockerhub.yml"
},
{
"@id": "github/workflows/linting.yml"
},
{
"@id": "github/workflows/awsfulltest.yml"
},
{
"@id": "github/workflows/awstest.yml"
},
{
"@id": "github/workflows/ci.yml"
},
{
"@id": "github/ISSUE_TEMPLATE/feature_request.md"
},
{
"@id": "github/ISSUE_TEMPLATE/bug_report.md"
},
{
"@id": "assets/nf-core-testpipeline_logo.png"
},
{
"@id": "assets/email_template.html"
},
{
"@id": "assets/multiqc_config.yaml"
},
{
"@id": "assets/email_template.txt"
},
{
"@id": "assets/sendmail_template.txt"
},
{
"@id": "conf/test_full.config"
},
{
"@id": "conf/base.config"
},
{
"@id": "conf/test.config"
},
{
"@id": "conf/igenomes.config"
}
],
"mainEntity": {
"@id": "nextflow.config"
}
},
{
"@id": "nextflow.config",
"@type": [
"File",
"Workflow",
"SoftwareSourceCode"
],
"programmingLanguage": {
"@id": "https://www.nextflow.io"
}
},
{
"@id": "CODE_OF_CONDUCT.md",
"@type": "File"
},
{
"@id": "LICENSE",
"@type": "File"
},
{
"@id": "environment.yml",
"@type": "File"
},
{
"@id": "CHANGELOG.md",
"@type": "File"
},
{
"@id": "Dockerfile",
"@type": "File"
},
{
"@id": "README.md",
"@type": "File"
},
{
"@id": "gitignore",
"@type": "File"
},
{
"@id": "gitattributes",
"@type": "File"
},
{
"@id": "nextflow_schema.json",
"@type": "File"
},
{
"@id": "main.nf",
"@type": "File"
},
{
"@id": "bin/markdown_to_html.py",
"@type": "File"
},
{
"@id": "bin/scrape_software_versions.py",
"@type": "File"
},
{
"@id": "docs/usage.md",
"@type": "File"
},
{
"@id": "docs/output.md",
"@type": "File"
},
{
"@id": "docs/README.md",
"@type": "File"
},
{
"@id": "docs/images/nf-core-testpipeline_logo.png",
"@type": "File"
},
{
"@id": "github/PULL_REQUEST_TEMPLATE.md",
"@type": "File"
},
{
"@id": "github/.dockstore.yml",
"@type": "File"
},
{
"@id": "github/CONTRIBUTING.md",
"@type": "File"
},
{
"@id": "github/markdownlint.yml",
"@type": "File"
},
{
"@id": "github/workflows/branch.yml",
"@type": "File"
},
{
"@id": "github/workflows/push_dockerhub.yml",
"@type": "File"
},
{
"@id": "github/workflows/linting.yml",
"@type": "File"
},
{
"@id": "github/workflows/awsfulltest.yml",
"@type": "File"
},
{
"@id": "github/workflows/awstest.yml",
"@type": "File"
},
{
"@id": "github/workflows/ci.yml",
"@type": "File"
},
{
"@id": "github/ISSUE_TEMPLATE/feature_request.md",
"@type": "File"
},
{
"@id": "github/ISSUE_TEMPLATE/bug_report.md",
"@type": "File"
},
{
"@id": "assets/nf-core-testpipeline_logo.png",
"@type": "File"
},
{
"@id": "assets/email_template.html",
"@type": "File"
},
{
"@id": "assets/multiqc_config.yaml",
"@type": "File"
},
{
"@id": "assets/email_template.txt",
"@type": "File"
},
{
"@id": "assets/sendmail_template.txt",
"@type": "File"
},
{
"@id": "conf/test_full.config",
"@type": "File"
},
{
"@id": "conf/base.config",
"@type": "File"
},
{
"@id": "conf/test.config",
"@type": "File"
},
{
"@id": "conf/igenomes.config",
"@type": "File"
}
]
}
See https://github.com/nf-core/tools/issues/686
PR checklist
- [x] This comment contains a description of changes (with reason)
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] Documentation in
docsis updated - [ ]
CHANGELOG.mdis updated - [ ]
README.mdis updated
Codecov Report
Merging #703 into dev will decrease coverage by
1.18%. The diff coverage is43.47%.
@@ Coverage Diff @@
## dev #703 +/- ##
==========================================
- Coverage 82.04% 80.85% -1.19%
==========================================
Files 13 14 +1
Lines 2746 2821 +75
==========================================
+ Hits 2253 2281 +28
- Misses 493 540 +47
| Impacted Files | Coverage Δ | |
|---|---|---|
| nf_core/ro_crate.py | 26.31% <26.31%> (ø) |
|
| nf_core/__main__.py | 62.36% <58.33%> (-0.33%) |
:arrow_down: |
| nf_core/lint.py | 92.06% <66.66%> (+0.37%) |
:arrow_up: |
| nf_core/utils.py | 90.00% <82.35%> (-1.12%) |
:arrow_down: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update c55255a...a42f004. Read the comment docs.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
Now that this proof-of-concept is working, I think we should prep a PR to the ro-crate library for some of this functionality. Then wait for a release there and then use it..
There's also a lot more metadata that we can add yet.
Could be useful: tutorial by @stain - https://biocompute-objects.github.io/bco-ro-crate/
Also a YouTube talk: https://youtu.be/3APqPwRIRkA
May now be redundant due to https://github.com/crs4/repo2crate
From what I can see, repo2crate is snakemake only?
From what I can see, repo2crate is snakemake only?
I haven't tried using it (only spied its existence in the WorkflowHub notes), but there's a nextflow.py file here? It's also listed under LANG_MODULES for the -l command line flag.
Hi, happy to join in this!
@simleo is trying in repo2crate to generalize what was before parsing just snakemake.
It picks up some nf-core metadata if the repository is well-formed following the nf-core guidelines (which would be the same for this tool, I guess).
It can help for registration in our https://workflowhub.eu/ and also in Simone's https://lifemonitor.eu/ of the Nf-core tests.
I am not sure if @simleo has added a pypi release that you could try use as a Python dependency and call its make_crate() function?
Hi everyone. I published a first 0.1.0 release of repo2rocrate (that's the name of the package and of the command line tool, I should rename the repository accordingly) to PyPI yesterday. The project did start as Snakemake only, then I added support for Galaxy (IWC) and Nextflow (nf-core). I've just finished updating the docs, hopefully they are enough to get started. For now there's nothing about the API, but it should be straightforward: users should call make_crate from the relevant language-specific module (this is shown in the CLI implementation).
Feedback is very welcome!
