Update `use_gitlab_ci()` for R packages, coverage and pkgdown

[statnmap]

Todo

• [x] Check package + pkgdown + coverage
• [x] Check package with {renv}
• [x] Build bookdown
• [ ] Build bookdown with {renv}
• [x] Build bookdown with dev and production versions
• [ ] Add Rbuildignore for "cache/", "ci/lib/" when running use_gitlab_ci()
• [ ] Add system dependencies sections
• [ ] Add badges to Readme.Rmd
• [ ] Add message (and command?) to retrieve coverage % on GitLab UI: Coverage: \d+\.\d+

See. https://gitlab.com/rdes_dreal/propre.rpls/-/blob/dev/.gitlab-ci.yml

[statnmap]

I think this would be interesting to create templates in "inst" with reduced version of the yaml below.
This would probably be easier to maintain than the current implementation of use_gitlab_ci().

Last complete version with check, main/dev versions of {pkgdown} + coverage, and global package cache:

image: rocker/geospatial

variables:
  R_LIBS_USER: "ci/lib"

cache:
  key: global-cache
  paths:
      - ci/lib/

stages:
  - build
  - test
  - pkgdown
  - pkgdown-move
  - deploy

building:
  stage: build
  script:
    - mkdir -p $R_LIBS_USER
    - apt update -qq && apt install -y libharfbuzz-dev libfribidi-dev
    - Rscript -e '.libPaths(c(Sys.getenv("R_LIBS_USER"), .libPaths()));remotes::install_deps(dependencies = TRUE, lib = Sys.getenv("R_LIBS_USER"))'
    - Rscript -e '.libPaths(c(Sys.getenv("R_LIBS_USER"), .libPaths()));remotes::install_cran(c("pkgdown"), lib = Sys.getenv("R_LIBS_USER"))'
    - Rscript -e '.libPaths(c(Sys.getenv("R_LIBS_USER"), .libPaths()));devtools::check()'

# To have the coverage percentage appear as a gitlab badge follow these
# instructions:
# https://docs.gitlab.com/ee/user/project/pipelines/settings.html#test-coverage-parsing
# The coverage parsing string is
# Coverage: \d+\.\d+

coverage:
    stage: test
    allow_failure: true
    when: on_success
    only:
        - master
        - dev
    script:
        - Rscript -e '.libPaths(c(Sys.getenv("R_LIBS_USER"), .libPaths()));covr::gitlab(quiet = FALSE);'
    artifacts:
        paths:
            - public
        expire_in: 30 days

# To produce a code coverage report as a GitLab page see
# https://about.gitlab.com/2016/11/03/publish-code-coverage-report-with-gitlab-pages/

# Create pkgdown for dev and master
integration:
    stage: pkgdown
    allow_failure: true
    when: on_success
    only:
        - master
        - dev
    script:
        - Rscript -e '.libPaths(c(Sys.getenv("R_LIBS_USER"), .libPaths()));remotes::install_local();pkgdown::build_site()'
    artifacts:
        paths:
            - docs
        expire_in: 30 days

# Move pkgdown for dev but recover previous master version if it exists
integration-dev:
    stage: pkgdown-move
    dependencies:
        - integration
    only:
        - dev
    script:
        - mkdir -p public/dev
        - wget https://gitlab.com/rdes_dreal/propre.antidote/-/jobs/artifacts/master/download?job=pages &&
          unzip -o download?job=pages &&
          rm download?job=pages &&
          echo "copied master artifacts" ||
          echo "copied master artifacts failed"
        - cp -r docs/* public/dev
    artifacts:
        paths:
            - public
        expire_in: 30 days

# Move pkgdown for master but recover previous dev version if it exists
integration-master:
    stage: pkgdown-move
    dependencies:
        - integration
    only:
        - master
    script:
        - mkdir -p public
        - wget https://gitlab.com/rdes_dreal/propre.antidote/-/jobs/artifacts/dev/download?job=pages &&
          unzip -o download?job=pages &&
          rm download?job=pages &&
          echo "copied dev artifacts" ||
          echo "copied dev artifacts failed"
        - cp -r docs/* public
    artifacts:
        paths:
            - public
        expire_in: 30 days

# Deploy pkgdown
pages:
    stage: deploy
    script:
        - echo "deployment with cache ?"
    artifacts:
        paths:
            - public
    only:
        - master
        - dev

Explore the effect of not "expire" artifacts. I think that after 30 days without development the public page disappear, but if nothing expires, maybe the server uses too much space for artifacts. OK

[statnmap]

For private project, this may require an access TOKEN to download artifacts : https://gitlab.com/profile/personal_access_tokens
https://docs.gitlab.com/ee/api/job_artifacts.html

JOB_TOKEN does not seem to work between branches. Better use your own TOKEN

- 'curl --location --output artifacts.zip --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/master/download?job=pages" &&
          unzip artifacts.zip &&
          rm artifacts.zip &&
          echo "copied master artifacts" ||
          echo "copied master artifacts failed"'

Don't use:

artifact_download:
  stage: test
  script:
    - 'curl --location --output artifacts.zip --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/master/download?job=test"'

or

artifact_download:
  stage: test
  script:
    - 'curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/master/download?job=test&job_token=$CI_JOB_TOKEN"'

But his is OK on private GitLab

        - 'curl --location --output artifacts.zip --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.private.fr/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/production/download?job=pages" &&
          unzip artifacts.zip &&
          rm artifacts.zip &&
          echo "copied production artifacts" ||
          echo "copied production artifacts failed"'

[statnmap]

With cache and corresponding Rprofile :

Ou --header "JOB-TOKEN: $CI_JOB_TOKEN" ?

The token has the same permissions to access the API as the user that triggers the pipeline. Therefore, this user must be assigned to a role that has the required privileges.

image: rocker/verse

variables:
  GIT_DEPTH: 10
  REPO_NAME: "https://packagemanager.rstudio.com/all/__linux__/focal/latest"
  R_LIBS_USER: "ci/lib"

cache:
  key: global-cache
  paths:
      - ci/lib/

stages:
  - build
  - test
  - pkgdown
  - pkgdown-move
  - deploy

building:
  stage: build
  script:
    # - apt-get update -qq && apt-get install -y libgdal-dev libproj-dev libgeos-dev libudunits2-dev libnode-dev libcairo2-dev libnetcdf-dev libharfbuzz-dev libfribidi-dev
    - apt-get update -qq && apt-get install -y libharfbuzz-dev libfribidi-dev
    - Rscript -e 'install.packages("remotes")'
    - Rscript -e 'install.packages("rcmdcheck")'
    # - Rscript -e 'remotes::install_github("rstudio/renv")'
    # - Rscript -e 'source("renv/activate.R")'
    # - Rscript -e 'options(repos = c(REPO_NAME = Sys.getenv("REPO_NAME")));renv::restore()'
    - Rscript -e 'remotes::install_local(upgrade = "never")'
    # - Rscript -e 'renv::install("pacman")'
    - R -e 'rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning", check_dir = "check")'

# To have the coverage percentage appear as a gitlab badge follow these
# instructions:
# https://docs.gitlab.com/ee/user/project/pipelines/settings.html#test-coverage-parsing
# The coverage parsing string is
# Coverage: \d+\.\d+

coverage:
    stage: test
    allow_failure: true
    when: on_success
    only:
        - master
        - dev
        - test-ci
    script:
        # - Rscript -e 'install.packages("pacman")'
        # - Rscript -e 'source("renv/activate.R")'
        # - Rscript -e 'options(repos = c(REPO_NAME = Sys.getenv("REPO_NAME")));renv::restore()'
        # - Rscript -e 'pacman::p_load("remotes");install_local()'
        - Rscript -e 'remotes::install_cran(c("covr", "DT"), upgrade = "never")'
        - Rscript -e 'covr::gitlab(quiet = FALSE)'
    artifacts:
        paths:
            - public
        expire_in: 30 days

# To produce a code coverage report as a GitLab page see
# https://about.gitlab.com/2016/11/03/publish-code-coverage-report-with-gitlab-pages/

integration:
    stage: pkgdown
    allow_failure: true
    when: on_success
    only:
        - master
        - dev
        - test-ci
    script:
        # - Rscript -e 'install.packages("pacman")'
        # - Rscript -e 'source("renv/activate.R")'
        # - Rscript -e 'options(repos = c(REPO_NAME = Sys.getenv("REPO_NAME")));renv::restore()'
        # - Rscript -e 'options(repos = c(REPO_NAME = Sys.getenv("REPO_NAME")));pacman::p_load("remotes");install_local()'
        - Rscript -e 'remotes::install_github("ThinkR-open/thinkrtemplate", upgrade = "never")'
        - Rscript -e 'remotes::install_cran(c("pkgdown"), upgrade = "never")'
        - Rscript -e 'pkgdown::build_site()'
    artifacts:
        paths:
            - docs
        expire_in: 30 days

integration-test:
    stage: pkgdown-move
    dependencies:
        - integration
    only:
        - test-ci
    script:
        - mkdir -p public/test
        - 'curl --location --output artifacts.zip --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.private.fr/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/master/download?job=pages" &&
          unzip artifacts.zip &&
          rm artifacts.zip &&
          echo "copied master artifacts" ||
          echo "copied master artifacts failed"'
        - 'curl --location --output artifacts.zip --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.private.fr/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/dev/download?job=pages" &&
          unzip artifacts.zip &&
          rm artifacts.zip &&
          echo "copied dev artifacts" ||
          echo "copied dev artifacts failed"'
        - cp -r docs/* public/test
    artifacts:
        paths:
            - public
        expire_in: 30 days

integration-dev:
    stage: pkgdown-move
    dependencies:
        - integration
    only:
        - dev
    script:
        - mkdir -p public/dev
        - 'curl --location --output artifacts.zip --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.private.fr/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/master/download?job=pages" &&
          unzip artifacts.zip &&
          rm artifacts.zip &&
          echo "copied master artifacts" ||
          echo "copied master artifacts failed"'
        - cp -r docs/* public/dev
    artifacts:
        paths:
            - public
        expire_in: 30 days

integration-master:
    stage: pkgdown-move
    dependencies:
        - integration
    only:
        - master
    script:
        - mkdir -p public
        - 'curl --location --output artifacts.zip --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.private.fr/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/dev/download?job=pages" &&
          unzip artifacts.zip &&
          rm artifacts.zip &&
          echo "copied dev artifacts" ||
          echo "copied dev artifacts failed"'
        - cp -r docs/* public
    artifacts:
        paths:
            - public
        expire_in: 30 days

pages:
    stage: deploy
    script:
        - echo "deployment with cache ?"
    artifacts:
        paths:
            - public
    only:
        - master
        - dev
        - test-ci

Rprofile

# For CI
# _CRAN mirror
if (Sys.getenv("REPO_NAME") != "") {
  options(repos = c(REPO_NAME = Sys.getenv("REPO_NAME")))
}

# _Cache packages
if (Sys.getenv("R_LIBS_USER") != "") {
  dir.create(Sys.getenv("R_LIBS_USER"), recursive = TRUE)
  .libPaths(c(Sys.getenv("R_LIBS_USER"), .libPaths()))
}

[MargotBr]

For library of packages management to avoid .libPaths(c(Sys.getenv("R_LIBS_USER"), .libPaths())) in .gitlab-ci.yml files :

Settings > CI/CD > Variables > R_LIBS = ci/lib

Add usethis::use_build_ignore("ci/lib")

In the .yml file :

image: rocker/verse

cache:
  key: global-cache
  paths:
      - $R_LIBS
    
stages:
  - build
  
building:
  stage: build
  script:
    - echo "Library path for packages :" $R_LIBS
    - mkdir -p $R_LIBS
    - Rscript -e 'install.packages(c("remotes"))'
    - Rscript -e 'remotes::install_deps(dependencies = TRUE, upgrade = "never")'
    - Rscript -e 'remotes::install_cran(c("here", "ggrepel", "thinkr", "rvg", "systemfonts", "gdtools", "officer", "uuid"), upgrade = "never")'
    - echo "Packages in library :" ls $R_LIBS
    - ls $R_LIBS
    - Rscript -e 'rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning", check_dir = "check")'

[statnmap]

• Maybe create a dedicated repository to store and download possible CI yml, like for github actions, so that you can have the latest yml, even without the latest {gitlabr} version
• Add badges in README if relevant
• Tell to the user about the activation of the coverage
• Maybe add a config file along with the YAML to define what to check and which badges

[statnmap]

• [x] Set variable inside the gitlab.ci

variables:
  GIT_DEPTH: 10
  REPO_NAME: "https://packagemanager.rstudio.com/all/__linux__/focal/latest"
  R_LIBS: "ci/lib"

[statnmap]

Add system requirements as for actions.

      - name: Query dependencies
        run: |
          install.packages('remotes')
          saveRDS(remotes::dev_package_deps(dependencies = TRUE), ".github/depends.Rds", version = 2)
          writeLines(sprintf("R-%i.%i", getRversion()$major, getRversion()$minor), ".github/R-version")
        shell: Rscript {0}

      - name: Install system dependencies
        if: runner.os == 'Linux'
        run: |
          while read -r cmd
          do
            eval sudo $cmd
          done < <(Rscript -e 'writeLines(remotes::system_requirements("ubuntu", "20.04"))')

Moved to https://github.com/statnmap/gitlabr/issues/49

[statnmap]

Add badges:

# _Badges
usethis::use_badge(badge_name = "pipeline status",
                   href = "https://<url>/<group>/<project>/-/commits/master",
                   src = "https://<url>/<group>/<project>/badges/master/pipeline.svg")
usethis::use_badge(badge_name = "coverage report",
                   href = "http://<group>.<pages-url>/<project>/coverage.html",
                   src = "https://<url>/<group>/<project>/badges/master/coverage.svg")

Note that pages-url is different than url Set default to gitlab.com

Moved to https://github.com/statnmap/gitlabr/issues/50