Filters don't work over 5000 options

[abresler]

Seems like after 5000 options the regex no longer works. Any chance we can override this? Here is a reproducible example using GDELT tags.

devtools::install_github("abresler/gdeltr2")
lapply(c("trelliscopejs", "gdeltr2", "purrr", "glue"), library, character.only = T)

  data <- 
    gdeltr2::get_gdelt_codebook_ft_api(code_book = "gkg") %>% 
    mutate_if(is.logical, as.character) %>%
    mutate(urlImage = "http://gdeltproject.org/imgs/gdelt-events-nasa-night-lights.jpg")
  
  parameter <- 'theme'
  data <-
    data %>%
    mutate(
      urlNews1D = glue::glue(
        'http://api.gdeltproject.org/api/v2/doc/doc?query={parameter}:{idGKGTheme}%20sourcelang:english&mode=artlist&timespan=24h&maxrecords=250&sort=datedesc'
      ),
      urlNews1M = glue::glue(
        'http://api.gdeltproject.org/api/v2/doc/doc?query={parameter}:{idGKGTheme}%20sourcelang:english&mode=artlist&timespan=1m&maxrecords=250&sort=datedesc'
      ),
      urlNews3M = glue::glue(
        'http://api.gdeltproject.org/api/v2/doc/doc?query={parameter}:{idGKGTheme}%20sourcelang:english&mode=artlist&timespan=12w&maxrecords=250&sort=datedesc'
      ),
      urlWordCloudEnglish3M = glue::glue(
        'http://api.gdeltproject.org/api/v2/doc/doc?query={parameter}:{idGKGTheme}%20sourcelang:english&mode=wordcloudenglish&timespan=12w&maxrecords=250&sort=datedesc'
      ),
      urlWordCloudTheme3M = glue::glue(
        'http://api.gdeltproject.org/api/v2/doc/doc?query={parameter}:{idGKGTheme}%20sourcelang:english&mode=wordcloudtheme&timespan=12w&maxrecords=250&sort=datedesc'
      ),
      urlTimelineVol = glue::glue(
        'http://api.gdeltproject.org/api/v2/doc/doc?query={parameter}:{idGKGTheme}%20sourcelang:english&mode=timelinevolinfo&timespan=12w&maxrecords=250&sort=datedesc'
      ),
      urlSentiment = glue::glue(
        'http://api.gdeltproject.org/api/v2/doc/doc?query={parameter}:{idGKGTheme}%20sourcelang:english&mode=ToneChart&timespan=12w&maxrecords=250&sort=datedesc'
      )
    )
  
  data <-
    data %>%
    mutate(idPanel = 1:n(),
           panel = trelliscopejs::img_panel(urlImage)) %>%
    mutate_at(
      data %>% dplyr::select(matches("^url")) %>% dplyr::select(-urlImage) %>% names(),
      funs(
        trelliscopejs::cog_href(., default_label = TRUE, default_active = TRUE)
      )
    )
  
  id_columns <-
    data %>% dplyr::select(matches("^id|^url")) %>% dplyr::select(-c(urlImage, idPanel, matches("idWBCode"))) %>% names()
  
  if (id_columns %>% length() == 1) {
    id_columns <- c('nameCodebook', id_columns)
  }
  
  title <-
    glue::glue("GDELT V2 API {parameter %>% str_to_upper()} CODES")
  
  data %>%
      trelliscopejs::trelliscope(
        name = title,
        nrow = 1,
        ncol = 3,
        state = list(
          labels = c(id_columns),
          sort = list(sort_spec("idPanel"))
        )
      )

[hafen]

I should be able to make this configurable. It's set to keep some bounds on performance, but an override for someone who knows the tradeoff should be reasonable.

In the case of the screenshot you share, where each value is unique per panel, it would probably be best in the future for the viewer to to provide a more powerful search box rather than a bar chart since the bar chart isn't interesting in this case.

[abresler]

Totally agree on some non barchart tool. Any ideas on what might work well?

---

Alex Bresler [email protected]

​www.asbcllc.com​

917-455-0239 (cell)

On Aug 8, 2017 6:00 PM, "hafen" [email protected] wrote:

I should be able to make this configurable. It's set to keep some bounds on performance, but an override for someone who knows the tradeoff should be reasonable.

In the case of the screenshot you share, where each value is unique per panel, it would probably be best in the future for the viewer to to provide a more powerful search box rather than a bar chart since the bar chart isn't interesting in this case.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hafen/trelliscopejs/issues/46#issuecomment-321094101, or mute the thread https://github.com/notifications/unsubscribe-auth/AAtM1LF2O3Nmdxh0AdE6tmIkLcW_33Bhks5sWNphgaJpZM4OrGTx .

[abresler]

I forked the package and overrode the threshold to 500,000 and it seems to work just fine!

Maybe we can pass a parameter to the workhorse trelliscopeJS function that lets the user define the threshold themselves but has a default of your 5000?

http://asbcllc.com/r_packages/gdeltr2/trelliscopes/v2_codebooks/gkg/index.html

[godi678]

I'm having the same problem with the regex search bar, I've forked the package but am not sure which file contains the code with the threshold?