Bioconductor support

[GuangchuangYu]

Someone reports a Windows specific issue of my Bioconductor package to me and I don't have Windows machine. I use check_on_windows to test it and it throw error for dependencies of Bioconductor package can't be resolved.

Is it possible to support Bioconductor? I believe the Bioconductor core team (@mtmorgan) would love to provide any assistance if needed.

  95#> > remotes::install_local('DOSE', dependencies = TRUE)

  96#> Skipping 24 packages not available: bitops, DBI, DO.db, fgsea, futile.logger, ggplot2, GO.db, irlba, knitr, magrittr, matrixStats, NMF, org.Hs.eg.db, plyr, reshape2, RSQLite, scales, snow, SparseM, testthat, tidyr, UpSetR, XML, xtable

  97#> Installing 18 packages: annotate, AnnotationDbi, Biobase, BiocGenerics, BiocParallel, BiocStyle, clusterProfiler, DOSE, GOSemSim, graph, GSEABase, igraph, IRanges, qvalue, Rcpp, RCurl, S4Vectors, topGO

  98#> Warning: unable to access index for repository https://cran.r-hub.io/bin/windows/contrib/3.3:

  99#> cannot open URL 'https://cran.r-hub.io/bin/windows/contrib/3.3/PACKAGES'

 100#> Warning: dependencies 'XML', 'DBI', 'xtable', 'RSQLite', 'futile.logger', 'snow', 'ggplot2', 'GO.db', 'magrittr', 'plyr', 'tidyr', 'DO.db', 'reshape2', 'scales', 'NMF', 'irlba', 'bitops', 'SparseM', 'matrixStats' are not available

[gaborcsardi]

Possible and desired.

As a first step we'll just need to add the BioC repos, I'll do that tomorrow(ish).

It would be also nice to have a local mirror of them.

[GuangchuangYu]

@mtmorgan maybe Bioconductor core team can help implementing checkBioC function which check the package with platform setting identical to BioC building servers. So that we can check our packages before committing to svn repo.

[gaborcsardi]

Actually, I added the BioC repos on Linux a long time ago, just forgot to do it on Windows: https://builder.r-hub.io/status/DOSE_3.1.1.tar.gz-566ff2227a214eaa9cdbeeaf928e339d

I'll add windows soon.

[gaborcsardi]

Took a "bit" longer, but finally I have added the BioC repos: https://builder.r-hub.io/status/graph_1.52.0.tar.gz-72775062a4324351b5abf5dd83be9cdd

[mtmorgan]

Bioconductor has a 'release' and a 'devel' branch. Most people building Bioconductor packages here would likely be doing so against the 'devel' branch (new features) rather than release branch (bug fixes). CRAN packages are a toss-up -- they might want to build against 'devel', since these are going to be relevant longer than 'release', or against 'release', since this is what users have access to now.

Bioconductor has a 6-month release cycle. As a consequence, our release cycle devel branch sometimes uses R-devel (as in the current Bioconductor release cycle), sometimes R-release (as in the next Bioconductor release cycle). This is summarized briefly on the Bioconductor web site.

In the example above the build is of the release version of a Bioconductor package, on the release branch of R. I guess that combination will always be 'correct', but package authors (usually, for Bioconductor package authors, often, for CRAN authors) will want the devel version of a Bioconductor package against the appropriate version of R.

[gaborcsardi]

@mtmorgan Thanks!

Is it appropriate if we use whatever BiocInstaller() chooses by default? (This is what happens currently.)

Optionally, users could choose bioc-release or bioc-devel explicitly, by setting a flag. Then we would just call useDevel(), as selected by the user. (Not implemented yet.)

Btw. does it ever happen that R-devel does not work with bioc-devel?

[mtmorgan]

When both release and devel Bioc are on release R, BiocInstaller defaults to release Bioc, so the developer needs some way to say 'no, use devel bioc'. On the other hand, when devel bioc is building on R-devel, it's an error to either ask to useDevel(TRUE) on release R, or useDevel(FALSE) on devel R.

When both release and devel Bioc are on release R, R-devel is 'unsupported' -- we don't build Bioc packages on R-devel, and individual packages can be broken because of changes in R. Changes to R haven't caused catastrophic changes to Bioc builds, but maybe ~5% of Bioc packages failed after our recent transition to R-devel. Usually these are not 'core' Bioc packages, because we tend to hear about changes in R that break core packages.

Thanks for your work on this, it's a great resource!

[gaborcsardi]

When both release and devel Bioc are on release R, BiocInstaller defaults to release Bioc, so the developer needs some way to say 'no, use devel bioc'.

Yes, that's what I meant by the flag.

Buy maybe we should approach this from the BioC version instead of the R version. I can add some shortcut functions to target bioc-release and bioc-devel on various platforms, and the appropriate R versions will be selected automatically and the useDevel() flag is also set appropriately. E.g.

check_with_bioc_release(os = c("linux", "windows", "macos"), ...)
check_with_bioc_devel(os = c("linux", "windows", "macos"), ...)

Does this make sense?

[lcolladotor]

I just linked to this thread from https://stat.ethz.ch/pipermail/bioc-devel/2018-February/012765.html. I think that Hervé Pagès has made a mapping of the Bioc versions and R versions somewhere.

Hm... I can't find the link on my bioc-list emails nor Hervé's github username.

[mtmorgan]

@lcolladotor http://bioconductor.org/about/release-announcements/ and https://bioconductor.org/config.yaml

[gaborcsardi]

@mtmorgan Thanks! The YAML file looks very nice!

[lcolladotor]

Awesome, thanks Martin! Those are the links I had in mind (but without enough clues for me to remember how to find them hehe).

[maelle]

A few notes (I'll update this comment)

• How to get the table R version - BioConductor version: https://github.com/Bioconductor/BiocManager/blob/0d509a650f33d437cd5eb2a597807384969d4ca3/R/version.R#L50 (a function that parses the YAML, with no dependency other than on utils). For some R versions, there will be two BioConductor versions, I guess that the highest one is the correct one.

• "When both release and devel Bioc are on release R, BiocInstaller defaults to release Bioc, so the developer needs some way to say 'no, use devel bioc'." (https://github.com/r-hub/rhub/issues/38#issuecomment-260456292) means a flag would still be useful.

• How to get repositories corresponding to a BioConductor version https://github.com/Bioconductor/BiocManager/blob/0d509a650f33d437cd5eb2a597807384969d4ca3/R/repositories.R#L30 (as in remotes now, does not seem to work for R versions prior to 3.1.0 i.e. the obtained repositories URLs give 404)

[maelle]

For reference here's what the YAML parsing function returns

config <- "https://bioconductor.org/config.yaml"
BiocManager:::.version_map_get_online(config)
#>    Bioc    R  BiocStatus
#> 1   1.6  2.1 out-of-date
#> 2   1.7  2.2 out-of-date
#> 3   1.8  2.3 out-of-date
#> 4   1.9  2.4 out-of-date
#> 5   2.0  2.5 out-of-date
#> 6   2.1  2.6 out-of-date
#> 7   2.2  2.7 out-of-date
#> 8   2.3  2.8 out-of-date
#> 9   2.4  2.9 out-of-date
#> 10  2.5 2.10 out-of-date
#> 11  2.6 2.11 out-of-date
#> 12  2.7 2.12 out-of-date
#> 13  2.8 2.13 out-of-date
#> 14  2.9 2.14 out-of-date
#> 15 2.10 2.15 out-of-date
#> 16 2.11 2.15 out-of-date
#> 17 2.12  3.0 out-of-date
#> 18 2.13  3.0 out-of-date
#> 19 2.14  3.1 out-of-date
#> 20  3.0  3.1 out-of-date
#> 21  3.1  3.2 out-of-date
#> 22  3.2  3.2 out-of-date
#> 23  3.3  3.3 out-of-date
#> 24  3.4  3.3 out-of-date
#> 25  3.5  3.4 out-of-date
#> 26  3.6  3.4 out-of-date
#> 27  3.7  3.5 out-of-date
#> 28  3.8  3.5     release
#> 29  3.9  3.6       devel
#> 30  3.9  3.7      future

^{Created on 2019-03-13 by the reprex package (v0.2.1)}

[maelle]

(I apologize for any error in the following)

One could imagine that the flag would be passed as an environment variable BIOCONDUCTOR_VERSION when using the hypothetical check_with_bioc_release() shortcuts.

E.g. the user calls check_with_bioc_release()

config <- "https://bioconductor.org/config.yaml"
info <- BiocManager:::.version_map_get_online(config)
r_version <- info$R[info$BiocStatus == "release"]
bioc_version <- info$Bioc[info$BiocStatus == "release"]
env_vars <- c(env_vars, "BIOCONDUCTOR_VERSION" = bioc_version)

In all platforms, when BIOCONDUCTOR_VERSION is "", let BiocManager do its things, but if it's not, call BiocManager::install(blabla, version = Sys.getenv("BIOCONDUCTOR_VERSION").

The problem is that r_version would be used by check_with_bioc_release() to select the proper platform by OS (r-devel vs r-release) but at the moment there's no way to know what version r-devel and r-release correspond to.

[maelle]

Reference on the BioConductor submission process http://bioconductor.org/developers/package-submission/#whattoexpect

[maelle]

Also, https://bioconductor.org/developers/package-guidelines/#checkingenv

[llrs]

Not sure if this is what you had in mind:

The problem is that r_version would be used by check_with_bioc_release() to select the proper platform by OS (r-devel vs r-release) but at the moment there's no way to know what version r-devel and r-release correspond to.

But isn't this what BiocVersion is intended to do?

---

I'm getting ready to submit a pacakge to CRAN with dependencies on Bioconductor I take the opportunity to help move this along

[gaborcsardi]

This issue is about the previous R-hub system, it does not apply to the new system, so I am closing it now. Please see https://r-hub.github.io/rhub/ for the new system, R-hub v2.