dockstore
nf-core github app - not installed on newer workflows - e.g. fetchngs
Describe the bug From GA4GH call, although we’ve been getting some nf-core updates up to roughly May
https://dockstore.org/organizations/nfcore/collections/pipelines
It seemed from bentsherman that some nf-core workflows were out-of-date.
Investigate whether this means the app hasn’t been installed on certain workflows, are we missing updates, has the app broken over time, etc.
Expected behavior Dockstore should have the same number of workflows as nf-core and with the same dates
Additional context
[Domain] - https://dockstore.org
- 1.15.2
- 2.12.2
- 1.15.0
┆Issue is synchronized with this Jira Story ┆Fix Versions: Dockstore 1.15.x ┆Issue Number: DOCK-2535 ┆Sprint: 141 - Narbo ┆Issue Type: Story
On closer inspection, it looks like the pipelines are up to date in Dockstore. I think what threw me off was that the "Engine Versions" reported by Dockstore are not consistent. For example, looking at rnaseq, Dockstore says "!>=22.10.1" but the latest version of rnaseq is "!>=23.04.0" :
https://github.com/nf-core/rnaseq/blob/b89fac32650aacc86fcda9ee77e00612a1d77066/nextflow.config#L324
Also, nf-core/fetchngs is missing which surprised me. Maybe it was created after you implemented the initial sync. So it might be worthwhile to review the https://nf-co.re/pipelines page for any prominent pipelines that are missing. That page can also tell you which ones are deprecated.
One more note about the Versions tab... going forward, I think it will be more useful to display the minimum Nextflow version instead of DSL1/2 for the "Language Versions" column. The DSL2 label will become increasingly useless because we add new features with every stable release, so you really need to know the minimum Nextflow version in order to run the pipeline.
Thanks, I've confirmed these and split out two different tickets. We'll get in touch with some nf-core folks to see what is going on with the app.
I think what is happening is the facets on the search page show engine versions for all of the workflows' versions. So while your latest version of rnaseq has !>=23.04.0, other versions of the workflow require different engines, and all the engine versions for all the workflow versions are displayed.
If you do this search, !>=23.04.0 is there, along with other engine versions.
But then we don't show you which workflow version requires which engine version in the UI when you get to workflow's page, so it's hard to tell. #5918 is a good idea.
TLDR: It looks like the app is indeed installed on newer workflows. For example https://github.com/nf-core/denovotranscript is two weeks old and shows up as https://dockstore.org/workflows/github.com/nf-core/denovotranscript:dev?tab=info The more prominent nextflow workflows seem unaffected. That said, there seems to be an issue with specific workflows, not https://dockstore.org/workflows/github.com/nf-core/fetchngs but workflows like reportho and demo. This has something to do with the pattern of users working on a workflow and whether they have accounts on Dockstore
Slightly longer version: There doesn't seem to be a widespread problem, for example reviewing released pipelines at https://nf-co.re/pipelines the first pipeline with a problem (sorted by stars) is https://nf-co.re/bactmap/1.0.0/ last updated 3 years ago. fetchngs in particular is present.
However, it does look like we have an issue with specific workflows that may have something to do with the users working on the workflow in question. For example, reportho and demo
This seems to be a variant of https://discuss.dockstore.org/t/user-does-not-have-an-account-on-dockstore-error/7691 or https://ucsc-cgl.atlassian.net/browse/SEAB-5994
Oh, I did notice a couple issues with nextflow workflows which failed builds like https://github.com/nf-core/genomeskim/tree/dev or workflows which have not been updated to dsl2 and are not compatible with newer nextflow versions, may split that out
Closing, I think the last issue was addressed with https://ucsc-cgl.atlassian.net/browse/SEAB-5994 Will re-open or spin-off tickets if we identify follow-up issues