blogdown 3.4 gitlab pages improvement

Hi Yihui, Amber & Alison,

I'm not completely sure of myself, but as it worked for me maybe it can help other people. I changed my gitlab-ci.yml from

image: rocker/tidyverse

before_script:
  - apt-get update
  - R -e "install.packages(c('blogdown', and_so_on), repos='https://cran.rstudio.com/')"
  - R -e "blogdown::install_hugo()"

pages:
  script:
  - R -e "blogdown::build_site()"
  artifacts:
    paths:
    - public
  only:
  - master

to the usual hugo one

image: monachus/hugo

variables:
  GIT_SUBMODULE_STRATEGY: recursive

pages:
  script:
  - hugo
  artifacts:
    paths:
    - public
  only:
  - master

and the deployment time dropped from 30 minutes to 30 seconds! Tidyverse and the other packages I needed are the longest to load. The new config seems to work for .rmd content as well because of their trailing .html, so it is a bit of a hack because this was probably not expected by you, developer ;)

Well I was thinking maybe it is worth a while to test the robustness of this config a bit further (I use wowchemy and icon hugo themes but don't have any sophisticated workflow) and let it know to the wide world.

Cheers Anna

Mar 24 '21 05:03 anna-doizy

Hi @anna-doizy,

Thanks a lot for sharing this! If both yaml will lead to same result (a hugo website) they use different ways, and main difference is that the first require R and the other only hugo. Here is a bit more context on the difference about this two valid workflows.

Basically, the second workflow will be the more efficient if you have already built all your Rmd file to .html or .md or .markdown and you only want your CI workflow to build the Hugo website. In that case, R is not required and having hugo will just work if you did not use specific bookdown features supported by build_site().

In fact, with last version, blogdown::build_site() will do very simple:

No build of any Rmd file by default - they are supposed to already been render. But it can be controlled by build_rmd
blogdown suppose custom build method with R/build.R and R/build2.R, respectively run before and after hugo build
It will tweak some parameters and environments variables for the Hugo build to help R users build their website with Hugo.

So yes, a blogdown project can have all its Rmd file build locally, then result .html (or other) published to git repo, and finally let Hugo only build the website.

I don't think this is a hack at all. You are using Hugo directly and this is perfectly fine if you know how to use it. blogdown is compatible with Hugo 100% and is here to help R user create a website without necessarily know about Hugo.

About the difference in timing, I believe this is because you are installing all the R packages without using a cache or a binary install. First, if you only want to build the site (blogdown::build_site()), no Rmd file are needed and you only need blogdown - so install.packages("blogdown") should be enough. This is especially TRUE with blogdown V1+ where Rmd files are no more rebuild by default. You need more R packages for you R Markdown file to be built if you want to rebuild some .Rmd files in your CI workflow. In that case I would advice to adapt the dependency installation to gain some installation time. Several solution:

Using a framework like renv that allow managing dependencies at project level and have a cache of package for quick restoring. it can be combine with the cache feature in Gitlab CI to only reinstall if you need too (generic example: https://rstudio.github.io/renv/articles/ci.html#gitlab-ci-1)
Use Linux binaries with RStudio Public Package Manager (RSPM) to make the installation quicker (this blog post is a good example for travis to adapt to gitlab CI)

Again, thanks a lot for sharing. This is very important for us ! This feedback gives us the incentive to document further the several possible solutions. We need to improve documentation and create example for different CI and build method. Thank you ! 👍

Mar 24 '21 08:03 cderv

Thank you so much, it is so nice to get such clear explanation!

I managed to decipher only part of it with a lot of trial and error and I'm really glad to know that it will be explained in this book. Actually, I think that my first yaml needed all these dependencies because I started with blogdown <1.0 and I didn't thought of changing that until yesterday.

I'm very glad that blogdown is fully compatible with Hugo because it allows me to work on websites with other people who don't use R at all while me keeping my beloved RStudio IDE ;)

Thanks again and good luck with the writing!

Mar 24 '21 10:03 anna-doizy

Actually, I think that my first yaml needed all these dependencies because I started with blogdown <1.0 and I didn't thought of changing that until yesterday.

Oh yeah before with <1.0.0 it would have been more painful. You can know more of the new stuff, especially regarding workflow in https://blog.rstudio.com/2021/01/18/blogdown-v1.0/

I'm very glad that blogdown is fully compatible with Hugo because it allows me to work on websites with other people who don't use R at all while me keeping my beloved RStudio IDE ;)

With new RStudio IDE 1.4 there is a new visual editor (https://blog.rstudio.com/2020/09/30/rstudio-v1-4-preview-visual-markdown-editing/) and it is really useful for non R user too because it makes writing RMarkdown or just Markdow file really easy !

Thanks again and good luck with the writing!

Thank you - The difficult part is to make time to improve the docs and examples !

Mar 24 '21 11:03 cderv