Highlight Important Text

[ClaudioZandonella]

In a personal email, an enthusiastic trackdown user suggested a very interesting idea:

Highlighting/colouring text that should not be changed in google docs (e.g. inline code, code chunks etc.).

In this way, we can minimize the risk that other users unintentionally modify these parts while editing the narrative text.

Great idea! I have started working on it, it is a little bit tricky but it is doable. I open this thread to document the development of this new feature.

Good news: Google provides API for document editing:

• google API format text https://developers.google.com/docs/api/how-tos/format-text
• batchUpdate https://developers.google.com/docs/api/reference/rest/v1/documents/batchUpdate
• updateTextStyleRequest https://developers.google.com/docs/api/reference/rest/v1/documents/request#UpdateTextStyleRequest
• TextStyle https://developers.google.com/docs/api/reference/rest/v1/documents#TextStyle

Bad news: r package googledrive (on which trackdown is based to manage google API) does not support Google Docs specific API (i.e., "https://docs.googleapis.com").

I have created a project for trackdown in the Google cloud platform enabling the required API. trackdown works as usual plus I was able to edit document text using the API. During the download, the formatting is automatically removed so we do not have to care about it.

However, the project on the Google cloud platform is currently in testing status (only invited users can use it). Before to published it, we need to follow best practices to deal with authentication.

• googledrive functions for authorization https://googledrive.tidyverse.org/reference/index.html#authorization
• gargle functions to create API request https://gargle.r-lib.org/reference/request_develop.html
• gargle manage API authentication for own service https://gargle.r-lib.org/articles/get-api-credentials.html
• gargle other functions https://gargle.r-lib.org/reference/oauth_app_from_json.html

To summarize, this feature is possible but requires creating a separate profile on the Google cloud platform with the required API. Plus we need to be sure that the whole authentication process is correctly managed. Plus understanding quotas and the number of operations allowed.

I will try first to ask at googledrive r package maintainers if they can introduce support for Google Docs specific API.

Finally, here is an example of an API request.

# build request text style

body <- list(requests = list(
  updateTextStyle = list(
    range = list(
      startIndex = 1,
      endIndex = 50
    ),
    textStyle = list(
      backgroundColor = list(
        color = list(
          rgbColor = list(
            blue = .99,
            green = .57,
            red = .62
          )
        )
      )
    ),
    fields = "*"
  )
),
writeControl = NULL)

req <- gargle::request_build(
  method = "POST",
  path = "v1/documents/{fileId}:batchUpdate",
  params = list(fileId = "<file-Id>"),
  body = body,
  base_url = "https://docs.googleapis.com",
  token = googledrive::drive_token()
)

resp <- gargle::request_make(req)
out <- gargle::response_process(resp)

[ClaudioZandonella]

Sorry for the time it took me. But finally, here we are with some good updates!

Now a beta version of the rich_text features is ready. The idea is that important text that should not be changed (e.g., code chunks and inline code) is highlighted in the Docs document. Here is an example of how it looks:

At the moment this works only for Rmd files.

The feature is available in the rich-text branch. You can install it with

remotes::install_github("ClaudioZandonella/trackdown", ref = "rich-text")

Next both upload_file() and update_file() functions have by default rich_text = TRUE. To change highlighting color see the example in the functions documentation.

There is only one limit at the moment. Now, trackdown uses its own API credentials (OAuth client ID and secret). Therefore, if you try to use the new version of the package you need to authenticate again. However, the trackdown project on Google Cloud Platform is public but it still requires verification. Thus, you will find a scary window. Don't worry, click "advanced" and "Go to Trackdown R Package (unsafe)"

Next, select all the options to allow upload/edit files.

The Trackdown R Package does nothing dangerous, I followed the same system as googledrive package that was previously used. You can find all detail on privacy etc. at https://claudiozandonella.github.io/trackdown/articles/trackdown-privacy-policy.html

I will request verification as soon as everything is ready. I was just too excited to share this new feature.

This is still in development and I am sure there will be lots of strange issues. However, you can already use it!

Soon I will post new updates.

[ClaudioZandonella]

Now the rich text feature is available on the master branch. Simply install it with,

remotes::install_github("ClaudioZandonella/trackdown")

Few notes:

• both .Rmd and .Rnw are supported
• I started the Google verification process of the App

[arthur-albuquerque]

It works very smoothly on my PC, a game-changer! Thanks, Claudio.

If I may suggest, it would be very nice if {trackdown} also highlighted citation brackets (e.g., [@cite]) throughout the text, as it does for inline R code (r ... ). These citation brackets are commonly used (it is natively supported in RStudio https://rstudio.github.io/visual-markdown-editing/citations.html).

Thank you again!

[ClaudioZandonella]

Hi @arthur-albuquerque,

Brilliant idea!!

I have implemented highlighting for citations (only for rmd not for rnw files). I matched the @cit-tag (without parentheses) so also in-text citations are highlighted.

At the moment changes are in the develop branch.

I add highlighting for equations as well and then I will merge in the master

Thanks again for your suggestion!

[ClaudioZandonella]

Merged version 1.3.4 into the master branch. Now, citations tags (@citation-tag), in-line equations ($math equation$) and equation blocks ($$math equation$$) are automatically highlighted.

[jobreu]

Thanks for the amazing package and for adding the cool new rich-text feature! I tested it a bit and noticed that sometimes not all characters are highlighted correctly on GDocs. For example, the triple backticks at the start of a code chunk and the opening { were not highlighted for some of the code chunks when I tested it, and sometimes the first two letters of plain text following the closing triple backticks at the end of a code chunk were also highlighted. I checked my underlying .Rmd file but was unable to identify why this may be the case.

[ClaudioZandonella]

Hi @jobreu,

Thanks for pointing out this problem. Really strange behaviour.

To highlight, we match the text using regular expressions. Do you mind sharing the document or part of the document to reproduce the problem? if you prefer, you can send it directly to my email ([email protected]).

Hopefully, the problem is simply in the regex. However, given the strange behaviour, it could also be something related to character encoding (maybe there are some invisible characters that shift the matching). I will try to figure out the problem

[jobreu]

Thanks a lot for getting back to me about this so quickly! :-) I have sent you an e-mail with the files attached. My first guess was also that it may be related to the regex, but from looking at my example I was not able to spot the pattern.

[stahl-c]

I also had the problem that the highlights are sometimes shifted by one character. It only occurred in later parts of the document, not in earlier sections, and it disappeared in the later parts after I removed a special character (°) from the document.

[ClaudioZandonella]

Hi @stahl-c,

thanks for pointing out the problem and thanks for the good intuition about the special character causing the issue👍. This will help a lot to fix the problem.

Unfortunately, it will take me a couple of weeks before I can work again on trackdown (recently I finished my PhD, found a new job and moved to another city; crazy busy weeks). I need a few more weeks to arrange everything and then I will be finally able to get back on developing trackdown.

[krlmlr]

With trackdown_auth(), I see:

This app is blocked

This app tried to access sensitive info in your Google Account. To keep your account safe, Google blocked this access.

I'm using a business Google account. Is there a way to fix this?

CC @maelle.

[krlmlr]

I followed https://claudiozandonella.github.io/trackdown/articles/oauth-app-configuration.html and set up the authentication successfully.

I also realized that this issue is about a new feature but not about generic auth problems. What's a good place to discuss auth?

[ClaudioZandonella]

Hi @krlmlr

Thank you for pointing out this need, absolutely a good idea.

I just created an issue where to collect all authentication problems. I pointed out the configuration instructions.

https://github.com/ClaudioZandonella/trackdown/issues/59

I am happy everything worked for you and that you were able to set up the authentication successfully.

Please do not hesitate to indicate other possible or possible suggestions❤️

[maelle]

@ClaudioZandonella you can now hide these comments as "off-topic" :innocent: (not requesting this, just mentioning this in case you didn't know about the feature!)