board_s3() expose ServerSideEncryption and SSEKMSKeyId parameters

[michkam89]

Hi, is it possible to use board_s3() with encrypted buckets? I couldn't find a way to pass ServerSideEncryption and SSEKMSKeyId parameters.

Recently similar issue was for targets package here, it would be great to have similar solution here :)

[juliasilge]

Related to #628

If we pass the dots through to the paws put_object method, this would support ServerSideEncryption and SSEKMSKeyId. It would look like:

board_sales <- board_s3("company-pins", prefix = "sales/")
board_sales %>% pin_write(mtcars, ServerSideEncryption = "AES256", SSEKMSKeyId = "your-key-here")

[juliasilge]

Closed in #648

Let us know if you have trouble with this moving forward!

[fh-mthomson]

Wow, what great timing! I was just search for how to specify encryption via board_s3() and pin_write() in order to move to a pins-based workflow away from our current aws.s3 functions. Thank you @juliasilge for this!

Unfortunately I'm hitting issues when trying to run the example above after installing the version on main at 7379792150e6b99cc3704f93e585c085aee67252:

board %>% pin_write(mtcars, ServerSideEncryption = "AES256")
board %>% pin_write(mtcars, Tagging = "key1=value1&key2=value2")

Both yield analogous error about unused dots:

Using `name = 'mtcars'`
Guessing `type = 'rds'`
Creating new version '20220909T233142Z-b8faa'
Error: AccessDenied (HTTP 403). Access Denied
Error in `pin_store()`:
! Arguments in `...` must be used.
✖ Problematic argument:
• ServerSideEncryption = "AES256"
Run `rlang::last_error()` to see where the error occurred.
Error in `pin_store()`:
! Arguments in `...` must be used.
✖ Problematic argument:
• ServerSideEncryption = "AES256"
Run `rlang::last_error()` to see where the error occurred.
Error in `pins::pin_write()`:
! Arguments in `...` must be used.
✖ Problematic argument:
• ServerSideEncryption = "AES256"
Run `rlang::last_error()` to see where the error occurred.

More detailed trace:

> rlang::last_error()
<error/rlib_error_dots_unused>
Error in `pins::pin_write()`:
! Arguments in `...` must be used.
✖ Problematic argument:
• ServerSideEncryption = "AES256"
---
Backtrace:
 1. board %>% pins::pin_write(mtcars, ServerSideEncryption = "AES256")
 2. pins::pin_write(., mtcars, ServerSideEncryption = "AES256")
Run `rlang::last_trace()` to see the full context.
> rlang::last_trace()
<error/rlib_error_dots_unused>
Error in `pins::pin_write()`:
! Arguments in `...` must be used.
✖ Problematic argument:
• ServerSideEncryption = "AES256"
---
Backtrace:
    ▆
 1. ├─board %>% pins::pin_write(mtcars, ServerSideEncryption = "AES256")
 2. └─pins::pin_write(., mtcars, ServerSideEncryption = "AES256")
 3.   └─rlang (local) `<fn>`()
 4.     └─rlang:::check_dots(env, error, action, call)
 5.       └─rlang:::action_dots(...)
 6.         ├─base (local) try_dots(...)
 7.         └─rlang (local) action(...)

[fh-mthomson]

For reference, the analogous approach we use via aws.s3 (thank you @jgutman for the initial implementation!)

opts <- list()
opts$headers <- c(`x-amz-server-side-encryption` = "AES256")
aws.s3::s3write_using(
  mtcars,
  readr::write_csv,
  object = s3_path,
  bucket = bucket,
  opts = opts
)

[juliasilge]

@fh-mthomson Can you extra-double-check that you have installed from GitHub? Maybe try removing the package and reinstalling? I see results like this:

library(pins)
b <- board_s3(bucket = "pins-testing-julia", profile = "PowerUser-158990117310", region = "us-east-2")
b %>% pin_write(mtcars, name = "more-nice-cars", ServerSideEncryption = "AES256")
#> Guessing `type = 'rds'`
#> Creating new version '20220910T002340Z-66143'
#> Writing to pin 'more-nice-cars'
b %>% pin_write(mtcars, name = "more-nice-cars", Tagging = "key1=value1&key2=value2")
#> Guessing `type = 'rds'`
#> Creating new version '20220910T002340Z-66143'
#> Writing to pin 'more-nice-cars'

^{Created on 2022-09-09 with reprex v2.0.2}

[fh-mthomson]

I tried several permutations to no avail, unfortunately:

• Inside / outside of projects with renv activated
• devtools::install_github("rstudio/pins-r@main"), devtools::install_github("rstudio/pins-r")
• Building a cloned version of the repo and loading

At this point I am fairly confident that I am using the development version:

devtools::package_info("pins")
#>  package   * version    date (UTC) lib source
#>  askpass     1.1        2019-01-13 [1] CRAN (R 4.1.0)
#>  cli         3.4.0      2022-09-08 [1] CRAN (R 4.1.3)
#>  curl        4.3.2      2021-06-23 [1] CRAN (R 4.1.0)
#>  digest      0.6.29     2021-12-01 [1] CRAN (R 4.1.0)
#>  ellipsis    0.3.2      2021-04-29 [1] CRAN (R 4.1.0)
#>  fansi       1.0.3      2022-03-24 [1] CRAN (R 4.1.2)
#>  filelock    1.0.2      2018-10-05 [1] CRAN (R 4.1.0)
#>  fs          1.5.2      2021-12-08 [1] CRAN (R 4.1.0)
#>  glue        1.6.2      2022-02-24 [1] CRAN (R 4.1.2)
#>  httr        1.4.4      2022-08-17 [1] CRAN (R 4.1.2)
#>  jsonlite    1.8.0      2022-02-22 [1] CRAN (R 4.1.2)
#>  lifecycle   1.0.2      2022-09-09 [1] CRAN (R 4.1.3)
#>  magrittr    2.0.3      2022-03-30 [1] CRAN (R 4.1.2)
#>  mime        0.12       2021-09-28 [1] CRAN (R 4.1.0)
#>  openssl     2.0.2      2022-05-24 [1] CRAN (R 4.1.2)
#>  pillar      1.8.1      2022-08-19 [1] CRAN (R 4.1.2)
#>  pins        1.0.2.9000 2022-09-10 [1] Github (rstudio/pins-r@7379792)
#>  pkgconfig   2.0.3      2019-09-22 [1] CRAN (R 4.1.0)
#>  R6          2.5.1      2021-08-19 [1] CRAN (R 4.1.0)
#>  rappdirs    0.3.3      2021-01-31 [1] CRAN (R 4.1.2)
#>  rlang       1.0.5      2022-08-31 [1] CRAN (R 4.1.3)
#>  sys         3.4        2020-07-23 [1] CRAN (R 4.1.0)
#>  tibble      3.1.8      2022-07-22 [1] CRAN (R 4.1.2)
#>  utf8        1.2.2      2021-07-24 [1] CRAN (R 4.1.0)
#>  vctrs       0.4.1      2022-04-13 [1] CRAN (R 4.1.2)
#>  whisker     0.4        2019-08-28 [1] CRAN (R 4.1.0)
#>  withr       2.5.0      2022-03-03 [1] CRAN (R 4.1.2)
#>  yaml        2.3.5      2022-02-21 [1] CRAN (R 4.1.2)
#>  zip         2.2.1      2022-09-08 [1] CRAN (R 4.1.3)
#> 
#>  [1] /Users/michael.thomson/Library/R/x86_64/4.1/library
#>  [2] /Library/Frameworks/R.framework/Versions/4.1/Resources/library
pins:::s3_upload_file
#> function (board, key, path, ...) 
#> {
#>     ellipsis::check_dots_used()
#>     body <- readBin(path, "raw", file.size(path))
#>     board$svc$put_object(Bucket = board$bucket, Key = paste0(board$prefix, 
#>         key), Body = body, ...)
#> }
#> <bytecode: 0x7fcdc922f658>
#> <environment: namespace:pins>

^{Created on 2022-09-09 by the reprex package (v2.0.1)}

[fh-mthomson]

Ah! It's an issue with the ... not being passed through to s3_upload_yaml() here: https://github.com/rstudio/pins-r/blob/main/R/board_s3.R#L224.

After fixing this on a local branch, I was able to successfully write with encryption. I couldn't push a feature branch to create a PR directly in pins-r so I created a PR in a forked version here: https://github.com/rstudio/pins-r/pull/652

> board %>%
+   pin_write(
+     mtcars,
+     ServerSideEncryption = 'AES256'
+   )
Using `name = 'mtcars'`
Guessing `type = 'rds'`
Creating new version '20220910T023000Z-b8faa'
Writing to pin 'mtcars'

[github-actions[bot]]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.