Reticulate Rprofile segfault

[kdkavanagh]

trafficstars

In my RProfile.site, I've got a few lines to setup my reticulate environment. If I launch R directly, this works fine. If I launch radian with --no-site-file and then manually source the file, this also works fine. However, if I launch radian reading from RProfile.site, I get a segfault. My guess is that there is some reticulate initialization that radian does that hasnt yet run by the time R reads RProfile?

from RProfile:

if( "reticulate" %in% utils::installed.packages() ) {
        reticulate::use_virtualenv('~/git/tools/.venv', required=TRUE)
        reticulate::py_run_string('from x.y.z import Config')
}

Output

(.venv): ~/git/tools $ radian

 *** caught segfault ***
address 0x10, cause 'memory not mapped'

Traceback:
 1: py_initialize(config$python, config$libpython, config$pythonhome,     config$virtualenv_activate, config$version >= "3.0", interactive(),     numpy_load_error)
 2: doTryCatch(return(expr), name, parentenv, handler)
 3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 4: tryCatchList(expr, classes, parentenv, handlers)
 5: tryCatch({    py_initialize(config$python, config$libpython, config$pythonhome,         config$virtualenv_activate, config$version >= "3.0",         interactive(), numpy_load_error)}, error = function(e) {    Sys.setenv(PATH = oldpath)    if (is.na(curr_session_env)) {
  Sys.unsetenv("R_SESSION_INITIALIZED")    }    else {        Sys.setenv(R_SESSION_INITIALIZED = curr_session_env)    }    stop(e)})
 6: initialize_python(required_module, use_environment)
 7: ensure_python_initialized()
 8: reticulate::py_run_string("from x.y.z import Config")

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 2

Session info

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server 7.7 (Maipo)

Matrix products: default
BLAS:   /usr/lib64/libblas.so.3.4.2
LAPACK: /usr/lib64/liblapack.so.3.4.2

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.6.1       Matrix_1.2-18        tools_3.6.1
[4] Rcpp_1.0.5           reticulate_1.14-9001 grid_3.6.1
[7] data.table_1.13.0    jsonlite_1.7.0       lattice_0.20-41

Python:

$ python --version
Python 3.6.4

[randy3k]

If I launch radian with --no-site-file and then manually source the file, this also works fine.

I don't think it will work fine. radian would force the reticulate to use the python environment where radian is installed at. reticulate::use_virtualenv will become malfunction.

My guess is that there is some reticulate initialization that radian does that hasnt yet run by the time R reads RProfile?

Because radian is written in python, we could only use the current python environment. (radian also prompt to switch radian if a different runtime is specified)

[kdkavanagh]

That's okay - The use_venv() path matches the venv used to launch* python/radian, so not concerned about removing that if necessary. Segfault occurs on following line tho (from x.y.x import...)

[kdkavanagh]

Just tested - Removed the use_venv and still see segault on following line

[randy3k]

It is expected to go wrong, because reticulate::py_run_string will invoke some versions of python which is not the current runtime. The advice is not to put any reticulate code for radian.

[kdkavanagh]

Hmm not sure I follow. My process is to activate a venv from bash which contains radian + some other libs I use via reticulate. Then from within R/radian I call those libs using reticulate. I've since removed the explicit "reticulate::use_virtualenv". The net result is that reticulate and radian should at least be using the same venv/py executable. If you could point me to where the rprofile gets sourced by radian/rchitect, I'd be happy to dig into it further

This works totally fine when I execute it by hand inside an existing radian terminal using that same venv but without the RProfile; it only fails when included in RProfile.site. Is your suggestion that radian is not compatible with end-user reticulate usage in general?

[randy3k]

The net result is that reticulate and radian should at least be using the same venv/py executable.

It is not true. As I have said earlier, reticulate::py_run_string in .Rprofile will invoke some versions of python which is not the current runtime. The reason that reticulate:: py_run_string works with --no-site-file is that radian has done some setup after R loads .Rprofile.

[kdkavanagh]

I see what you mean now - thanks. Do you mind pointing me to that addtl setup logic?

[randy3k]

It is here https://github.com/randy3k/radian/blob/master/radian/radianapp.py#L117