taxa icon indicating copy to clipboard operation
taxa copied to clipboard

HTTP errors when parsing long taxon_id list

Open janstrauss1 opened this issue 5 years ago • 12 comments

Hi there,

I'm trying to create a taxmap from a long list of NCBI taxon IDs for subsequent filtering.

I have downloaded about 17k taxa containing a specific protein domain from InterPro and imported into R

my.tax_id <- read.table(file = "TaxID_IPR012674.txt")
> str(my.tax_id)
'data.frame':	17482 obs. of  1 variable:
 $ V1: int  104 158 162 166 17 172 192 195 196 197

I then try to set um my taxmap as follows:

my.taxmap <- lookup_tax_data(
  tax_data = my.tax_id, 
  type = "taxon_id", 
  column = 1, 
  datasets = list(),
  mappings = c(), 
  database = "ncbi", 
  include_tax_data = TRUE,
  use_database_ids = TRUE, 
  ask = TRUE
  )
Looking up classifications for 17482 unique taxon IDs from database "ncbi"...

Unfortunately, this throws the error Error: Too Many Requests (HTTP 429)

I guess the API client is making too many concurrent requests to the database which causes the error.

Could you please help to fix it?

Many thanks in advance!

The output of sessionInfo() is

R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] urltools_1.7.3 taxize_0.9.91  taxa_0.3.2    

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.3        pillar_1.4.2      compiler_3.6.1    plyr_1.8.4        iterators_1.0.12  tools_3.6.1      
 [7] jsonlite_1.6      tibble_2.1.3      nlme_3.1-141      lattice_0.20-38   pkgconfig_2.0.3   rlang_0.4.1      
[13] foreach_1.4.7     cli_1.1.0         rstudioapi_0.10   crul_0.9.0        curl_4.2          parallel_3.6.1   
[19] dplyr_0.8.3       stringr_1.4.0     xml2_1.2.2        triebeard_0.3.0   grid_3.6.1        tidyselect_0.2.5 
[25] reshape_0.8.8     glue_1.3.1        httpcode_0.2.0    data.table_1.12.6 R6_2.4.1          reshape2_1.4.3   
[31] purrr_0.3.3       magrittr_1.5      codetools_0.2-16  assertthat_0.2.1  bold_0.9.0        ape_5.3          
[37] stringi_1.4.3     crayon_1.3.4      zoo_1.8-6  

janstrauss1 avatar Nov 25 '19 16:11 janstrauss1