drive_upload entire folders?

[svenhalvorson]

I'm not sure if this is possible or not but it doesn't seem like I can just choose a local directory and upload all the contents to google drive.

Is this possible or in the works?

[jennybc]

It's a workflow we plan to write an article about (#123, #25) in the near term.

Longer term, we could add a recursive argument to drive_upload(), but then it becomes more distant from the endpoint it wraps. The Drive API doesn't allow folder upload, so we'd have to do all the folder creation and file looping from R.

[tiernanmartin]

Glad to hear that you're considering documenting this in an article!

So there's no way to upload folders at all through the Drive API?

Does that explain why I'm seeing this sort of error when I run code like this?

drive_upload(media = "/folder01/",  
             path = "~/projects/", 
             type = "folder")
#> Error in curl::curl_fetch_memory(url, handle = handle) : 
#>   A libcurl function was given a bad argument

[jennybc]

Does that explain why I'm seeing this sort of error when I run code like this?

Probably. We should try to detect the attempt to upload a folder and give a more informative message.

So there's no way to upload folders at all through the Drive API?

AFAIK yes. You can upload files. You can create files, including directories. So to get something that feels like you are uploading a local directory to Drive, we need to do the individual pieces of work ourselves and wrap it for you.

[tiernanmartin]

I see - so I could conceivably put together a workflow that :

1. replicates my project directory in Drive using drive_mkdir() and then
2. uploads the local files to their corresponding Drive folders

Sound about right?

[jennybc]

Yep!

[tarunparmar]

Hi @tiernanmartin , if you have a routine you mentioned above, do you mind sharing it ? Thanks!

[FeMike82]

I too am interested. I did a quick lookup and found these articles. I'll read and see how to use them

http://technodocbox.com/C_and_CPP/73108864-Package-googledrive-r-topics-documented-august-28-2017.html

And

https://www.rdocumentation.org/packages/googledrive/versions/0.1.2/topics/drive_mkdir

[tiernanmartin]

Hi @tarunparmar and @FeMike82

I never did develop that workflow. I guess that the need to programmatically upload folders to Drive hasn't come up enough in my work to justify spending the time to create and troubleshoot this routine.

Sorry!

[tarunparmar]

No worries, I was able to make use of drive_find to get the folder id and use that to create sub folder using drive_mkdir followed by drive_upload.

Just need to put these in a loop to run on batch of files.

[kendavidn]

Did anyone manage to write a workflow or function to do this yet? Direly needed for my not-very-good-with-loops self.

[seth-cp]

I've written a function to upload a folder to Google Drive recursively. Let me know if this is of use to anyone! @kendavidn

#' Upload a folder to Google Drive
#'
#' Upload the contents of a folder (directory) to Google Drive recursively. The
#' implementation is depth-first. Only uploads objects that have type "file" or
#' "directory" according to `fs::dir_info()`; ignores types "symlink", "FIFO",
#' "socket", "character_device" or "block_device".
#'
#' @param folder The local folder that is to be uploaded, given as a path e.g.
#'   with `fs::path()`. Note only the contents of the folder are uploaded; the
#'   original local folder's name will not appear in Google Drive.
#'
#' @param drive_path The destination folder on Google Drive, given as a URL, file id, or dribble
#'
#' @return A dribble of the uploaded files (not directories though.)
drive_upload_folder <- function(folder, drive_path) {
  # Only call fs::dir_info once in order to avoid weirdness if the contents of the folder is changing
  contents <- fs::dir_info(folder, type = c("file", "dir"))
  dirs_to_upload <- contents %>%
    dplyr::filter(type == "directory") %>%
    pull(path)

  # Directly upload the files
  uploaded_files <- contents %>%
    dplyr::filter(type == "file") %>%
    pull(path) %>%
    purrr::map_dfr(googledrive::drive_upload, path = drive_path)

  # Create the next level down of directories
  dirs_to_upload %>%
    fs::path_rel(folder) %>%
    purrr::map(., googledrive::drive_mkdir, path = drive_path) %>%
    # Recursively call this function
    purrr::map2_dfr(dirs_to_upload, ., drive_upload_folder) %>%
    # return a dribble of what's been uploaded
    dplyr::bind_rows(uploaded_files) %>%
    invisible()
}

[duncan-clark]

I used this function - works great!

[bryan506]

1. made my desired directories on google drive (which can also be done with drive_mkdir()).
2. copy/paste folder ID
3. wrote a for loop to iterate through my local directory and upload to Gdrive.

drive_mkdir(...) googledrive::drive_find(n_max = 5) for (paths in list.files(path = "path", pattern = ".csv", full.names = TRUE)) { drive_upload(paths, path = as_id("ID")) }

I'm using KDE on Manjaro and use KioGdrive to mount my Gdrive folders so I can also just create from there.