ee_Initialize() tries to authenticate even if already done?

[MatthieuStigler]

• rgee version: 1.1.5
• R version: 4.2.0
• Operating System: Ubuntu 22.04 server

At submit an issue, please attached the following information of your rgee session:

• [x] You have the Python API installed (from terminal):

• [x] You can find the credentials file on your system:

• [ ] You can run a simple EE command from R: NO

Attach your Python (reticulate) configuration:

library(reticulate)
py_config()
python:         /home/user/anaconda3/envs/ee/bin/python
libpython:      /home/user/anaconda3/envs/ee/lib/libpython3.8.so
pythonhome:     /home/user/anaconda3/envs/ee:/home/user/anaconda3/envs/ee
version:        3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:10)  [GCC 10.3.0]
numpy:           [NOT FOUND]
ee:             /home/user/anaconda3/envs/ee/lib/python3.8/site-packages/ee

Description

When running ee_Initialize() rgee tries to authenticate the session even if my session is already authenticate. In theory this should not be a big issue as I could go to the authentication steps again, but as I am running on a server, this makes it much more complicated.

How is rgee detecting whether the sessions is already authenticated or not?

ee_install_set_pyenv(py_path = "~/anaconda3/envs/ee/bin/python", py_env = "ee")
system("~/anaconda3/envs/ee/bin/earthengine ls") ## works
ee_Initialize(user = '[email protected]', gcs = FALSE, drive = FALSE)
── rgee 1.1.5 ───────────────────────────────────────────────────────────────────────────────────── earthengine-api 0.1.317 ── 
 ✔ user: [email protected] 
 ✔ Initializing Google Earth Engine:Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=517222506229-**************************

[treetexan]

I would like to +1 this issue--I had some bugs and got Google's help in setting up my authentificate, but rgee is bouncing and unable to run it the way they specified. Would be a huge help if this could be disabled.

[treetexan]

also, not sure if this is a good idea, but with a working python earthengine authenticate, I was able to circumvent ee_Initialize() and just run rgee::ee$Initialize() which it is a wrapper for.

[MatthieuStigler]

oh nice, thanks! So you are just running rgee::ee$Initialize(), without any option/argument, right? Do you have a sense of what this means regarding the gcs or drive arguments in ee_Initialize?

[csaybar]

Hi @MatthieuStigler and @treetexan, sorry for the inconvenience!

did you try ee_Authenticate? #293

I'm working on upgrading the docker container (if it works there, it must work in a server too). I will come back here when I solve it.

[MatthieuStigler]

Unfortunately, the new ee_Authenticate with auth_quiet will not solve it. I am in a situation where:

• system("~/anaconda3/envs/ee/bin/earthengine ls"): :heavy_check_mark: works, meaning I did the authentication
• rgee::ee$Initialize(): :heavy_check_mark: works
• ee_Initialize(user = xxx): :x: does NOT work, and tries to do the authentication again

So it looks like rgee is looking somewhere for credentials, and doesn't find them? How can we check that?

Below are my env variables:

Sys.getenv("EARTHENGINE_PYTHON")
[1] "~/anaconda3/envs/ee/bin/python"
> Sys.getenv("EARTHENGINE_ENV")
[1] "ee"

[MatthieuStigler]

Mmh, maybe the problem is the user to authenticate? I was doing rgee::ee_Initialize(auth_quiet = TRUE) but then trying rgee::ee_Initialize(user="username") in my scripts, which would not work.

Trying now rgee::ee_Initialize(user="username", auth_quiet = TRUE) it works, but it seems unable to save the conf file? I see the warning:

In file.create(to[okay]) : cannot create file '~/.config/earthengine//username/credentials', reason 'Not a directory'

As a consequence, running just after rgee::ee_Initialize(user = '[email protected]') will try again to authenticate :-(

rgee::ee_Initialize(user = '[email protected]', auth_quiet = TRUE) 
── rgee 1.1.5 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── earthengine-api 0.1.317 ── 
 ✔ user: [email protected] 
 ✔ Initializing Google Earth Engine:Fetching credentials using gcloud
You are authorizing client libraries without access to a web browser. ...

...

Credentials saved to file: [~/.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).

Successfully saved authorization token.
 ✔ Initializing Google Earth Engine:  DONE!
 ✔ Earth Engine account: users/username 
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
Warning message:
In file.create(to[okay]) :
  cannot create file '~/.config/earthengine//username/credentials', reason 'Not a directory'

[csaybar]

r u on server side isn't it?

[MatthieuStigler]

yes, all commands are run on server side (except for the two-step deferred gcloud auth)

[csaybar]

this is weird...

In file.create(to[okay]) : cannot create file '~/.config/earthengine//username/credentials', reason 'Not a directory' do

do you have permission to write? maybe creating the folder before could solve the problem, btw I can't reproduce the error.

[MatthieuStigler]

ok, I see what happened: ~/.config/earthengine//username was actually a file, not a directory, which triggered the error above.

Not sure how this happened, that file was a private-key I believe (see pseudo-output). So maybe I did a mistake once and wrongly put this file under this name...

Anyway, removing that file now works, rgee::ee_Initialize(user = '[email protected]', auth_quiet = TRUE) is now permanent. So the issue can be closed, though we might want to wait for @treetexan to confirm the problem is also solved on their side?

THanks again @csaybar for your kind help!

[treetexan]

Hi all

Thanks for hammering away at this! I am watching my young kids all weekend but will reply tmrw with more.

In my setup the .config file is read only, and I can’t undo that (tried but it reverts). But R had no problem creating an empty username folder. Thing is, the credentials folder is stored in /.config/eathengine when you create the credentials file via the auth_mode=notebook route. There is no username subfolder. Copying the credentials into a username subfolder I create before did not work.

Thanks Matt

On Sun, Oct 2, 2022 at 7:43 AM Cesar Aybar @.***> wrote:

this is weird...

In file.create(to[okay]) : cannot create file '~/.config/earthengine//username/credentials', reason 'Not a directory' do

do you have permission to write? maybe creating the folder before must solve the problem, I can't reproduce the error.

— Reply to this email directly, view it on GitHub https://github.com/r-spatial/rgee/issues/291#issuecomment-1264623199, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJAJL5EEYJPD3TXEDNAIFV3WBFYNNANCNFSM6AAAAAAQPMARLY . You are receiving this because you were mentioned.Message ID: @.***>

[treetexan]

this is what I got--what the solution i tested (third line of code) what you were asking me to do?

library(rgee) options(error = traceback) rgee::ee_Initialize(user = @.', auth_quiet = TRUE) ── rgee 1.1.5 ──────────────────────────────────── earthengine-api 0.1.324 ── ✔ user: @. ✔ Initializing Google Earth Engine:Error in (function (authorization_code = NULL, code_verifier = NULL, auth_mode = NULL, : unused argument (auth_quiet = TRUE) No traceback available db <- 'CGIAR/SRTM90_V4' image <- ee$Image(db) Error in py_call_impl(callable, dots$args, dots$keywords) : ee.ee_exception.EEException: Earth Engine client library not initialized. Run ee.Initialize() image$bandNames()$getInfo() Error in image$bandNames : object of type 'closure' is not subsettable rgee::ee$Initialize() db <- 'CGIAR/SRTM90_V4' image <- ee$Image(db) image$bandNames()$getInfo() [1] "elevation"

On Sun, Oct 2, 2022 at 8:57 AM Matthew F @.***> wrote:

Hi all

Thanks for hammering away at this! I am watching my young kids all weekend but will reply tmrw with more.

In my setup the .config file is read only, and I can’t undo that (tried but it reverts). But R had no problem creating an empty username folder. Thing is, the credentials folder is stored in /.config/eathengine when you create the credentials file via the auth_mode=notebook route. There is no username subfolder. Copying the credentials into a username subfolder I create before did not work.

Thanks Matt

On Sun, Oct 2, 2022 at 7:43 AM Cesar Aybar @.***> wrote:

this is weird...

In file.create(to[okay]) : cannot create file '~/.config/earthengine//username/credentials', reason 'Not a directory' do

do you have permission to write? maybe creating the folder before must solve the problem, I can't reproduce the error.

— Reply to this email directly, view it on GitHub https://github.com/r-spatial/rgee/issues/291#issuecomment-1264623199, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJAJL5EEYJPD3TXEDNAIFV3WBFYNNANCNFSM6AAAAAAQPMARLY . You are receiving this because you were mentioned.Message ID: @.***>

[treetexan]

Wrote too soon, got it fixed! Same code, but I first copied my credentials file into the @.*** subfolder, and now it runs. Note that rgee created that subfolder in the last set of code I shared (it didn't previously exist and wasn't created by the credentials authorization process in Python), so this required code failing first to work.

Thank you both! I am very happy to not be fighting this bug anymore.

library(rgee) rgee::ee_Initialize(user = @.', auth_quiet = TRUE) ── rgee 1.1.5 ──────────────────────────────────── earthengine-api 0.1.324 ── ✔ user: @. ✔ Initializing Google Earth Engine: ✔ Initializing Google Earth Engine: DONE!

✔ Earth Engine account: users/mfagan ─────────────────────────────────────────────────────────────────────────────

db <- 'CGIAR/SRTM90_V4' image <- ee$Image(db) image$bandNames()$getInfo() [1] "elevation"

On Mon, Oct 3, 2022 at 3:09 PM Matthew F @.***> wrote:

this is what I got--what the solution i tested (third line of code) what you were asking me to do?

library(rgee) options(error = traceback) rgee::ee_Initialize(user = @.', auth_quiet = TRUE) ── rgee 1.1.5 ──────────────────────────────────── earthengine-api 0.1.324 ── ✔ user: @. ✔ Initializing Google Earth Engine:Error in (function (authorization_code = NULL, code_verifier = NULL, auth_mode = NULL, : unused argument (auth_quiet = TRUE) No traceback available db <- 'CGIAR/SRTM90_V4' image <- ee$Image(db) Error in py_call_impl(callable, dots$args, dots$keywords) : ee.ee_exception.EEException: Earth Engine client library not initialized. Run ee.Initialize() image$bandNames()$getInfo() Error in image$bandNames : object of type 'closure' is not subsettable rgee::ee$Initialize() db <- 'CGIAR/SRTM90_V4' image <- ee$Image(db) image$bandNames()$getInfo() [1] "elevation"

On Sun, Oct 2, 2022 at 8:57 AM Matthew F @.***> wrote:

Hi all

Thanks for hammering away at this! I am watching my young kids all weekend but will reply tmrw with more.

In my setup the .config file is read only, and I can’t undo that (tried but it reverts). But R had no problem creating an empty username folder. Thing is, the credentials folder is stored in /.config/eathengine when you create the credentials file via the auth_mode=notebook route. There is no username subfolder. Copying the credentials into a username subfolder I create before did not work.

Thanks Matt

On Sun, Oct 2, 2022 at 7:43 AM Cesar Aybar @.***> wrote:

this is weird...

In file.create(to[okay]) : cannot create file '~/.config/earthengine//username/credentials', reason 'Not a directory' do

do you have permission to write? maybe creating the folder before must solve the problem, I can't reproduce the error.

— Reply to this email directly, view it on GitHub https://github.com/r-spatial/rgee/issues/291#issuecomment-1264623199, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJAJL5EEYJPD3TXEDNAIFV3WBFYNNANCNFSM6AAAAAAQPMARLY . You are receiving this because you were mentioned.Message ID: @.***>

[MatthieuStigler]

great, so it seems the issue was solved for both of us, I'll close it then