asdfree
asdfree copied to clipboard
Published
20 hours ago •
ajdamico
ajdamico
Problem downloading annual PNAD files
Hello! I am unable to download the annual PNAD files using the very same code available here: http://asdfree.com/pesquisa-nacional-por-amostra-de-domicilios-pnad.html
Below the code with the result I get:
> pnad_cat <-
+ get_catalog( "pnad" ,
+ output_dir = file.path( path.expand( "~" ) , "PNAD" ) )
building catalog for pand
>
> # 2011 only
> pnad_cat <- subset( pnad_cat , year == 2011 )
> # download the microdata to your local computer
> pnad_cat <- lodown( "pnad" , pnad_cat )
locally downloading pnad
'ftp://ftp.ibge.gov.br/Trabalho_e_Rendimento/Pesquisa_Nacional_por_Amostra_de_Domicilios_anual/microdados/reponderacao_2001_2012/PNAD_reponderado_2011_20150814.zip'
cached in
'/var/folders/f4/_03f10nd691_9dy9qz0xy2n40000gn/T//edf5264e01900e39c02009a5b4ab60a8.Rcache'
copying to
'/var/folders/f4/_03f10nd691_9dy9qz0xy2n40000gn/T//RtmpRu78tj/file512373d1814'
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.5
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
locale:
[1] pt_BR.UTF-8/pt_BR.UTF-8/pt_BR.UTF-8/C/pt_BR.UTF-8/pt_BR.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] rio_0.5.10 forcats_0.3.0 stringr_1.3.1 dplyr_0.7.6 purrr_0.2.5 readr_1.1.1
[7] tidyr_0.8.1 tibble_1.4.2 ggplot2_3.0.0 tidyverse_1.2.1 lodown_0.1.0
loaded via a namespace (and not attached):
[1] tidyselect_0.2.4 reshape2_1.4.3 haven_1.1.2 lattice_0.20-35 colorspace_1.3-2 yaml_2.1.19
[7] blob_1.1.1 rlang_0.2.2 pillar_1.2.2 foreign_0.8-70 glue_1.3.0 withr_2.1.2
[13] DBI_1.0.0 selectr_0.4-1 bit64_0.9-7 modelr_0.1.2 readxl_1.1.0 bindrcpp_0.2.2
[19] bindr_0.1.1 plyr_1.8.4 munsell_0.5.0 gtable_0.2.0 cellranger_1.1.0 rvest_0.3.2
[25] memoise_1.1.0 psych_1.8.4 parallel_3.5.0 curl_3.2 broom_0.4.4 Rcpp_0.12.18
[31] scales_1.0.0 jsonlite_1.5 bit_1.1-14 mnormt_1.5-5 digest_0.6.16 hms_0.4.2
[37] openxlsx_4.0.17 stringi_1.2.4 grid_3.5.0 bitops_1.0-6 cli_1.0.0 tools_3.5.0
[43] magrittr_1.5 RCurl_1.95-4.11 lazyeval_0.2.1 RSQLite_2.1.1 crayon_1.3.4 pkgconfig_2.0.1
[49] data.table_1.11.4 xml2_1.2.0 lubridate_1.7.4 assertthat_0.2.0 httr_1.3.1 rstudioapi_0.7
[55] R6_2.2.2 nlme_3.1-137 compiler_3.5.0
lodown is now exiting unexpectedly.
websites that host publicly-downloadable microdata change often and sometimes those changes cause this software to break.
if the error call stack below appears to be a hiccup in your internet connection, then please verify your connectivity and retry the download.
otherwise, please open a new issue at `https://github.com/ajdamico/asdfree/issues` with the contents of this error call stack and also the output of your `sessionInfo()`.
[[1]]
lodown("pnad", pnad_cat)
[[2]]
withCallingHandlers(catalog <- load_fun(data_name = data_name,
catalog, ...), error = function(e) {
print(sessionInfo())
if (grepl("cannot allocate vector of size", e))
message(memory_note)
else if (grepl("parameter must be specified", e))
message(parameter_note)
else if (grepl("to install", e))
message(installation_note)
else {
message(unknown_error_note)
print(sys.calls())
}
})
[[3]]
load_fun(data_name = data_name, catalog, ...)
[[4]]
unzip_warn_fail(tf, exdir = paste0(tempdir(), "/unzips"))
[[5]]
tryCatch({
unzip(...)
}, warning = function(w) stop(conditionMessage(w)))
[[6]]
tryCatchList(expr, classes, parentenv, handlers)
[[7]]
tryCatchOne(expr, names, parentenv, handlers[[1L]])
[[8]]
value[[3L]](cond)
[[9]]
stop(conditionMessage(w))
[[10]]
.handleSimpleError(function (e)
{
print(sessionInfo())
if (grepl("cannot allocate vector of size", e))
message(memory_note)
else if (grepl("parameter must be specified", e))
message(parameter_note)
else if (grepl("to install", e))
message(installation_note)
else {
message(unknown_error_note)
print(sys.calls())
}
}, "erro -1 na extração a partir de arquivo zip", quote(value[[3L]](cond)))
[[11]]
h(simpleError(msg, call))
Error in value[[3L]](cond) : erro -1 na extração a partir de arquivo zip
year
12 2011
ftp_folder
12 ftp://ftp.ibge.gov.br/Trabalho_e_Rendimento/Pesquisa_Nacional_por_Amostra_de_Domicilios_anual/microdados/reponderacao_2001_2012/
sas_ri
12 <NA>
full_url
12 ftp://ftp.ibge.gov.br/Trabalho_e_Rendimento/Pesquisa_Nacional_por_Amostra_de_Domicilios_anual/microdados/reponderacao_2001_2012/PNAD_reponderado_2011_20150814.zip
output_filename case_count
12 /Users/chris.duartec/PNAD/2011 main.rds NA
Also, the results of sessionInfo()
> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.5
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
locale:
[1] pt_BR.UTF-8/pt_BR.UTF-8/pt_BR.UTF-8/C/pt_BR.UTF-8/pt_BR.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] rio_0.5.10 forcats_0.3.0 stringr_1.3.1 dplyr_0.7.6 purrr_0.2.5 readr_1.1.1
[7] tidyr_0.8.1 tibble_1.4.2 ggplot2_3.0.0 tidyverse_1.2.1 lodown_0.1.0
loaded via a namespace (and not attached):
[1] tidyselect_0.2.4 reshape2_1.4.3 haven_1.1.2 lattice_0.20-35 colorspace_1.3-2 yaml_2.1.19
[7] blob_1.1.1 rlang_0.2.2 pillar_1.2.2 foreign_0.8-70 glue_1.3.0 withr_2.1.2
[13] DBI_1.0.0 selectr_0.4-1 bit64_0.9-7 modelr_0.1.2 readxl_1.1.0 bindrcpp_0.2.2
[19] bindr_0.1.1 plyr_1.8.4 munsell_0.5.0 gtable_0.2.0 cellranger_1.1.0 rvest_0.3.2
[25] memoise_1.1.0 psych_1.8.4 parallel_3.5.0 curl_3.2 broom_0.4.4 Rcpp_0.12.18
[31] scales_1.0.0 jsonlite_1.5 bit_1.1-14 mnormt_1.5-5 digest_0.6.16 hms_0.4.2
[37] openxlsx_4.0.17 stringi_1.2.4 grid_3.5.0 bitops_1.0-6 cli_1.0.0 tools_3.5.0
[43] magrittr_1.5 RCurl_1.95-4.11 lazyeval_0.2.1 RSQLite_2.1.1 crayon_1.3.4 pkgconfig_2.0.1
[49] data.table_1.11.4 xml2_1.2.0 lubridate_1.7.4 assertthat_0.2.0 httr_1.3.1 rstudioapi_0.7
[55] R6_2.2.2 nlme_3.1-137 compiler_3.5.0
What is happening? Thank you!
this script (lodown) uses a HTTR package that is not working with IBGEs ftp server. I´ve ran on similar issue (trying to download other survey). As far as I know, this hasn't been fixed.
hi! apologies for the long delay. i've made a couple of big updates to asdfree.com that hopefully make the website a bit better, but i've decided to stop maintaining the lodown package so probably won't fix the bug you've reported. the new asdfree does have pnadc data, but only for the most current year. thanks
