Collect.reddit fails hard on large threads

[Karel-Kroeze]

I'm aware that collecting comments from reddit is limited to 500 comments, but I did not expect the collection to error and exit completely. Is this intentional?

# First we create the authentication token and then we collect the data. 
reddit.data <- Authenticate("reddit") |>
              Collect(
                threadUrls = "https://www.reddit.com/r/Music/comments/pgm7le/abba_will_release_a_new_album_first_time_since/",
                verbose = TRUE)

results in:

Error: Error in `dplyr::mutate()`:
! Problem while computing `rm = ifelse(...)`.
Caused by error in `ifelse()`:
! dims [product 1] do not match the length of object [574]
7.   stop(gsub("^Error:\\s", "", paste0(e)), call. = FALSE)
6.   value[[3L]](cond)
5.   tryCatchOne(expr, names, parentenv, handlers[[1L]])
4.   tryCatchList(expr, classes, parentenv, handlers)
3.   tryCatch({
        threads_df <- reddit_build_df(threadUrls, waitTime, ua, verbose, 
        msg = msg)
        }, error = function(e) { ...
2. Collect.reddit(Authenticate("reddit"), threadUrls ="https://www.reddit.com/r/Music/comments/pgm7le/abba_will_release_a_new_album_first_time_since/", 
verbose = TRUE)
1. Collect(Authenticate("reddit"), threadUrls = "https://www.reddit.com/r/Music/comments/pgm7le/abba_will_release_a_new_album_first_time_since/", 
verbose = TRUE)

I would expect collection to fail gracefully with either a warning or error message, and return whatever results it could obtain. If that is not possible, a more helpful error message would be nice.

[bryn-g]

Hi Karel,

That error is obviously not intentional. I believe it is a problem introduced with changes in the most recent version of the package that was not picked up in testing. Thank you kindly for providing the input threads so that I can reproduce and fix the issue in the next update.

[bryn-g]

This issue should hopefully be resolved now in dev version 0.32.8.

remotes::install_github("vosonlab/vosonSML")

> packageVersion("vosonSML")
[1] ‘0.32.8’

[Karel-Kroeze]

Thank you for the quick response and fix, I can confirm that at least for this particular thread the problem is fixed.

[maxevilmind]

+   Collect(threadUrls = myThreadUrls, writeToFile = F)
Error: Error in `dplyr::mutate()`:
! Problem while computing `rm = ifelse(...)`.
Caused by error in `ifelse()`:
! dims [product 1] do not match the length of object [742]

same here, fails when the number of comments is large

packageVersion("vosonSML")
[1] ‘0.32.8’

this fix did not work