Problem with user function that calls a second function

[JohannesNE]

If I make a function (fun1), call that function from another function (fun2), and call fun2 from a job, fun1 can not be found.

make_nums <- function() {
  rnorm(5)
}

sq_nums <- function() {
  make_nums()^2
}

sq_nums()
#> [1] 1.21640284 0.73830850 0.08544378 2.02388512 1.30504181

job::job({
  res <- sq_nums()
})
#> Error in make_nums() : could not find function "make_nums"
#> Calls: sourceWithProgress -> eval -> eval -> sq_nums
#> Execution halted

job::job({
  res <- sq_nums()
}, import = c(sq_nums, make_nums))
#> Error in make_nums() : could not find function "make_nums"
#> Calls: sourceWithProgress -> eval -> eval -> sq_nums
#> Execution halted

job::job({
  res <- make_nums()^2
}) 

res
#> [1] 0.0594687 1.9270315 0.1461293 0.3191069 0.1931480

Session info

─ Session info ─────────────────────────────────────────────────────────────────────
 setting  value                       
 version  R version 4.1.0 (2021-05-18)
 os       Pop!_OS 21.10               
 system   x86_64, linux-gnu           
 ui       RStudio                     
 language en_US:en                    
 collate  en_US.UTF-8                 
 ctype    en_US.UTF-8                 
 tz       Europe/Copenhagen           
 date     2022-02-25                  

─ Packages ─────────────────────────────────────────────────────────────────────────
 package     * version date       lib source        
 cachem        1.0.5   2021-05-15 [1] CRAN (R 4.1.0)
 callr         3.7.0   2021-04-20 [1] CRAN (R 4.1.0)
 cli           3.2.0   2022-02-14 [1] CRAN (R 4.1.0)
 crayon        1.5.0   2022-02-14 [1] CRAN (R 4.1.0)
 desc          1.4.0   2021-09-28 [1] CRAN (R 4.1.0)
 devtools      2.4.2   2021-06-07 [1] CRAN (R 4.1.0)
 ellipsis      0.3.2   2021-04-29 [1] CRAN (R 4.1.0)
 fastmap       1.1.0   2021-01-25 [3] CRAN (R 4.1.0)
 fs            1.5.2   2021-12-08 [1] CRAN (R 4.1.0)
 glue          1.6.2   2022-02-24 [1] CRAN (R 4.1.0)
 lifecycle     1.0.1   2021-09-24 [1] CRAN (R 4.1.0)
 magrittr      2.0.2   2022-01-26 [1] CRAN (R 4.1.0)
 memoise       2.0.0   2021-01-26 [1] CRAN (R 4.1.0)
 pkgbuild      1.3.1   2021-12-20 [1] CRAN (R 4.1.0)
 pkgload       1.2.1   2021-04-06 [1] CRAN (R 4.1.0)
 prettyunits   1.1.1   2020-01-24 [3] CRAN (R 4.1.0)
 processx      3.5.2   2021-04-30 [1] CRAN (R 4.1.0)
 ps            1.6.0   2021-02-28 [1] CRAN (R 4.1.0)
 purrr         0.3.4   2020-04-17 [3] CRAN (R 4.1.0)
 R6            2.5.1   2021-08-19 [1] CRAN (R 4.1.0)
 remotes       2.4.0   2021-06-02 [1] CRAN (R 4.1.0)
 rlang         1.0.1   2022-02-03 [1] CRAN (R 4.1.0)
 rprojroot     2.0.2   2020-11-15 [1] CRAN (R 4.1.0)
 sessioninfo   1.1.1   2018-11-05 [3] CRAN (R 4.1.0)
 testthat      3.0.4   2021-07-01 [1] CRAN (R 4.1.0)
 usethis       2.1.3   2021-10-27 [1] CRAN (R 4.1.0)
 withr         2.4.3   2021-11-30 [1] CRAN (R 4.1.0)

[1] /home/johannes/R/x86_64-pc-linux-gnu-library/4.1
[2] /usr/local/lib/R/site-library
[3] /usr/lib/R/site-library
[4] /usr/lib/R/library

[lindeloev]

Whoa, thank you for reporting! This seems like a very frequent use case.

First, here's a workaround until it's fixed:

Workaround option 1:

sq_nums <- function() {
  # "import" functions from calling environment
  make_nums2 = get("make_nums", envir = parent.frame())
  make_nums2()^2
}

Workaround option 2:

sq_nums <- function() {
  # Eval expression in calling environment
  eval(expression(make_nums()^2), envir = parent.frame())
}

The problem is that sq_norms look for make_nums in the global environment (globalenv()) but the main session is actually imported to an environment at level 3 inside the job::job() and this is where the code is executed (try print(sys.nframe()) in your main session and inside the job). This is actually what rstudioapi::jobRunScript() does.

[JohannesNE]

Thank you for creating and maintaining this package!

My use case is (of course) a bit more complicated, so I would prefer not to change the multiple layers of user functions to make the workaround work. No problem! Until it's fixed, I'll just get a cup of coffee while my console is occupied :)

[twest820]

+1 for this. job::job() is perfect for what I need to do. Unfortunately neither workaround works—the needed functions are in the global environment, which appears to be a couple levels up from the job parent, but remain inacessible even when specifying globalenv() or .GlobalEnv—and explicitly including functions with job::job(import) also fails despite 114.2 MB of other things getting imported.

Not a coffee drinker, so I guess either I'm making a lot of code changes or running as many R separate sessions as I can keep track of and merging the results using save() and load(). 😀

[lindeloev]

This bug seems to have resolved itself!? Possibly via an update of RStudio and/or rstudioapi. Can you confirm @JohannesNE and @twest820?

If it doesn't work with the CRAN version of job, then try remotes::install_github("lindeloev/job"). Compared to the CRAN version, it's just a bug-fix release. See NEWS.md.

[lindeloev]

job 0.3.1 is on CRAN now. I've added this "phenomenon" to the test suite and it passes on win/mac/linux. So I'm closing it for now. Please reopen if it persists!

[twest820]

Can you confirm @JohannesNE and @twest820?

Unfortunately I can't. This was enough of an adoption issue I never developed code relying on job. Cool that calls should flow now, though; I'll give it another try next time something in this direction comes up for coding in R (future does ok within it's use cases but I've found it tends to stop scaling around eight cores).