bigQueryR

bigQueryR copied to clipboard

Hi,

Running into an error when trying to upload data with bqr_upload_data after upgrading to R 4.0.2 on Ubuntu 20.04 bqr_upload_data(projectId = "XXX", datasetId = "XXX", tableId = "XXX", upload_data = upload_file, allowJaggedRows = TRUE, autodetect = FALSE, create = "CREATE_IF_NEEDED", writeDisposition = "WRITE_TRUNCATE")

Error messages: 2020-08-22 17:44:30 -- Uploading local data.frame 2020-08-22 17:44:31 -- Upload job made... 2020-08-22 17:44:36 -- Waiting for job: job_57rJOEQf4vFavmoccgYW-dAbPS4x - Job timer: 5.000396 secs Error in curl::curl_fetch_memory(url, handle = handle): Unrecognized content encoding type. libcurl understands identity content encodings. Request failed [ERROR]. Retrying in 1 seconds... Error in curl::curl_fetch_memory(url, handle = handle): Unrecognized content encoding type. libcurl understands identity content encodings. Request failed [ERROR]. Retrying in 1.1 seconds... Error in curl::curl_fetch_memory(url, handle = handle) : Unrecognized content encoding type. libcurl understands identity content encodings. Error: Request failed before finding status code: Unrecognized content encoding type. libcurl understands identity content encodings.

Thanks, could I see your sessionInfo()?

Thanks for the quick response!

`R version 4.0.2 (2020-06-22) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 20.04 LTS

Matrix products: default BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-openmp/libopenblasp-r0.3.8.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=C
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] teamr_0.0.1 RCurl_1.98-1.2 purrr_0.3.4 stringr_1.4.0 bigQueryR_0.5.0 googleCloudStorageR_0.5.1 [7] googleAuthR_1.3.0 dplyr_1.0.1 httr_1.4.2

loaded via a namespace (and not attached): [1] zip_2.1.0 Rcpp_1.0.5 pillar_1.4.6 compiler_4.0.2 bitops_1.0-6 tools_4.0.2 digest_0.6.25 jsonlite_1.7.0 memoise_1.1.0
[10] lifecycle_0.2.0 tibble_3.0.3 gargle_0.5.0 lattice_0.20-41 pkgconfig_2.0.3 rlang_0.4.7 Matrix_1.2-18 cli_2.0.2 rstudioapi_0.11 [19] curl_4.3 yaml_2.2.1 generics_0.0.2 vctrs_0.3.2 fs_1.5.0 askpass_1.1 hms_0.5.3 grid_4.0.2 tidyselect_1.1.0 [28] glue_1.4.1 R6_2.4.1 fansi_0.4.1 readr_1.3.1 magrittr_1.5 ellipsis_0.3.1 assertthat_0.2.1 stringi_1.4.6 openssl_1.4.2
[37] crayon_1.3.4`

Great thanks, and after looking at this issue https://github.com/jeroen/curl/issues/179 may I ask you re run the failing upload with options(googleAuthR.verbose=1) so we can examine the HTTP headers sent with the request and the response headers

This thread may help too https://github.com/jeroen/curl/issues/160

I only see now that despite the error my data is uploaded to BigQuery.

With verbose:

-> GET /bigquery/v2/projects/XXX/jobs/job_hr_FlMBJtoHi0qdBEJzyZhxZklC- HTTP/1.1 -> Host: www.googleapis.com -> User-Agent: googleAuthR/1.3.0 (gzip) -> Accept: application/json, text/xml, application/xml, / -> Accept-Encoding: gzip -> Authorization: Bearer XXX -> <- HTTP/1.1 200 OK <- ETag: YsspnhsRtTEmCaY40rNEwg== <- Content-Type: application/json; charset=UTF-8 <- Vary: Origin <- Vary: X-Origin <- Vary: Referer <- Content-Encoding: gzip <- Date: Mon, 24 Aug 2020 06:16:30 GMT <- Server: ESF <- Cache-Control: private <- X-XSS-Protection: 0 <- X-Frame-Options: SAMEORIGIN <- X-Content-Type-Options: nosniff <- Transfer-Encoding: chunked

I only see now that despite the error my data is uploaded to BigQuery.

Ok good info thanks, means its just the response that says "It worked" thats erroring.

And you say this only started recently when you upgraded from R 4.0 to R 4.0.2? Or what R were you on before, 3.7 ?

I raised an issue at curl to see if its something going on there.

@MarkEdmondson1234 I was on 3.6.3

@jurrigerretsen see the comment in the linked issue - comment there to be more direct :)

What version of libcurl are you using? Can you include your curl::curl_version() ? It seems like it was built without gzip support.