oauth2_proxy icon indicating copy to clipboard operation
oauth2_proxy copied to clipboard

Google - continually use refresh token ( NOT Working )

Open kishoreyellamraju opened this issue 8 years ago • 26 comments

The refresh access token is not happening on the master branch. The pull request #117 solved this but looks like its broken on the master branch. The session times out after 3600 seconds and asks user to login again.

kishoreyellamraju avatar Feb 23 '16 20:02 kishoreyellamraju

@kishore1431 can you share what your relevant config options are? Also, are you seeing any errors in the oauth2_proxy logs?

jehiah avatar Feb 23 '16 21:02 jehiah

Here are the config options that i am using. I don't see any errors but i no longer see the below statement in the logs

"2016/02/24 11:45:02 google.go:154: refreshed access token Session{[email protected] token:true expires:2016-02-24 12:45:02 -0800 PST refresh_token:true} (expired on 2016-02-24 11:21:58 -0800 PST)"

I rolled back to v2.0.1 and the refresh access token is working as expected but this version has "approval_prompt" hardcoded to "force" in the code :(.

Note: used dummy cookie/domain names.

approval_prompt="auto" access_type="offline" cookie_name = "_oauth2_abcdef_s1234" cookie_secret = "oauth2_1234oauth" cookie_domain = ".abcd.com" cookie_expire = "9h" cookie_refresh = "1h" cookie_secure = true

kishoreyellamraju avatar Feb 24 '16 20:02 kishoreyellamraju

@kishore1431 are you sure you are not just observing the timeout passed the 9h cookie TTL? (FWIW, i run with expiration set to 504h and refresh set to 3h.

jehiah avatar Feb 24 '16 20:02 jehiah

@jehiah :). Its definitely not the 9h timeout, its timing out every 60 mins and asking user to login again.

kishoreyellamraju avatar Feb 24 '16 21:02 kishoreyellamraju

Yes i have same issue. All cookie refresh is working, and there is valid response from google:

200 GET https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=<access_token> {
 "issued_to": "<id>.apps.googleusercontent.com",
 "audience": "<id>.apps.googleusercontent.com",
 "user_id": "<userid>",
 "scope": "https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email",
 "expires_in": 1108,
 "email": "[email protected]",
 "verified_email": true,
 "access_type": "offline"
}

in config:

cookie_refresh = "15m"
cookie_expire = "168h"

As we see in log output above, we have expire_in that google is setting to max 3600 seconds on start - no more, no less and it is not configurable as far as i know. And in this 1h period of time all is working like it should. When i set refresh cookie to 15 minutes and block user account in Google, then user gets 403 when cookie try to be refreshed and this is what we need, but......

2016/02/27 17:22:21 oauthproxy.go:533: 127.0.0.1:58466 ("111.111.111.111") refreshing 1h44m28s old session cookie for Session{[email protected] token:true expires:2016-02-27 16:37:52 +0000 UTC} (refresh after 15m0s)
2016/02/27 17:22:21 oauthproxy.go:547: 127.0.0.1:58466 ("111.111.111.111") removing session. token expired Session{[email protected] token:true expires:2016-02-27 16:37:52 +0000 UTC}

Last try to refresh cookies session failed because access token in Google expired and it is not refreshed in last 3600 seconds. We have feature like group access and refresh token in Google, but after this commit something is broken - https://github.com/bitly/oauth2_proxy/commit/d49c3e167f3dced5d8630be77dff5b6e25b4645c This code is never visible in logs as i check - https://github.com/bitly/oauth2_proxy/blob/d49c3e167f3dced5d8630be77dff5b6e25b4645c/providers/google.go#L154

Only way to start working again is refresh whole site in browser. This is especially bad idea when we operate on site with many javascripts and after 1 hour any request from javascript failing.

szibis avatar Feb 27 '16 21:02 szibis

More update. Looks like problem exists only on pages that use xhr requests from javascript. After one hour when access token expires and full auth is triggered for any request (cookie refresh and session expires in oauth2) then any request from javascript inside page fails. It fail on accounts.google.com, because Google is not allowing this origin to talk from javascript - CORS (they don't expose Access-Control-Allow-Origin header ). Even after adding Authorized JavaScript origins inside google developers pages settings for this oauth2 user.

I think main problem is that access token in oauth2 proxy is only redeemed when all sessions expired and triggers whole auth from web page side (using auth_request from nginx). This should be refreshed in background just like cookie refresh.

szibis avatar Mar 11 '16 08:03 szibis

Error from Google:

XMLHttpRequest cannot load https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&approval_p…callback&response_type=code&scope=profile+email&state=/. 
No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://<my origin> is therefore not allowed access.

Origin is available in request headers to Google. My origin is set in google developers panel in Authorized JavaScript origins, but as i write before only jsonp requests have Access-Control-Allow-Origin present from Google.

szibis avatar Mar 11 '16 10:03 szibis

Hi,

I believe we are seeing this also, is there a workaround or fix available for this?

r0ps3c avatar Apr 24 '16 15:04 r0ps3c

@szibis, where are you seeing that xhr error? In the logs for oauth2_proxy or somewhere else?

r0ps3c avatar May 09 '16 15:05 r0ps3c

Hi all. We also suffer from this in a heavily-Javascript based webapp. It's been really quiet on this bug for months - any progress / workarounds / whatever that can help move forward? We can code a bit too, but we don't know anything about Go. :)

antgel avatar Jul 19 '16 16:07 antgel

@antgel If you can confirm this is still a problem against the most recent release (v2.1) or HEAD and provide steps to reproduce that would be helpful.

jehiah avatar Jul 19 '16 17:07 jehiah

So, we are using the latest version with the auth_request, the users authenticate using a web page, but then they are redirected to a single page app and only use AJAX. And we see in the oauth2_proxy logs that it doesn't update their cookie, only removes it.

Reproduction is hard as our app is internal. Would you be up for doing a screenshare, or I could send screenshots or logs, or anything you tell me that would be useful. Sorry to be difficult but we would like to help.

I see lines in the log like: Jul 19 13:31:33 i-0e4e37ed80d3bbce2 oauth2_proxy[4424]: 2016/07/19 13:31:33 oauthproxy.go:538: 10.31.68.84:21097 refreshing 21h31m4s old session cookie for Session{[email protected] token:true expires:2016-07-18 17:00:29 +0000 UTC refresh_token:true} (refresh after 1h0m0s)

antgel avatar Jul 20 '16 16:07 antgel

I'm no longer seeing this bug (since after I originally reported it) but suspect that something else in our infrastructure - specifically outbound proxy filtering - was causing the issue

r0ps3c avatar Jul 20 '16 18:07 r0ps3c

We don't have anything fancy going on like that.

antgel avatar Jul 20 '16 19:07 antgel

@jehiah Any interest in a screenshare or similar as above? What sort of data would you need from us to reproduce the issue? Might logs help?

antgel avatar Jul 25 '16 13:07 antgel

@jehiah What else do you need from us? This is happening in all releases ( I tried all from versions in 2.x).

The refresh token is set to 3600 sec. whatever I do, it is expiring and timing out the user after 1 hour.

Here is my config,

cookie_name = "_oauth2_foo_bar" cookie_secret = "foobarfoobar==" cookie_domain = "foo.bar.com" cookie_expire = "9h" cookie_refresh = "1m" cookie_secure = true cookie_httponly = true

and the oauth2_proxy logs.

2017/04/21 17:35:28 oauthproxy.go:458: 10.111.227.113:44078 refreshing 4m17s old session cookie for Session{[email protected] token:true expires:2017-04-21 17:51:58 -0700 PDT} (refresh after 1m0s) 2017/04/21 17:35:28 internal_util.go:31: 200 GET https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=foo.barfoo-ep84O9jetIsfoobarfoobarfoobargspMpgzrcOxHKG6jfP9ki0LzcWpSgK-G5Bu3Zn-jhCLzGu_zYBlmCR8PZDn-qvh7lL2f73d8qJt5zHEnw { "issued_to": "2foobarfoobarfoobarnmsgn.apps.googleusercontent.com", "audience": "2foobarfoobarfoobarmsgn.apps.googleusercontent.com", "user_id": "1131391287foobar", "scope": "https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email", "expires_in": 990, "email": "[email protected]", "verified_email": true, "access_type": "offline" }

kishoreyellamraju avatar Apr 24 '17 20:04 kishoreyellamraju

Someone mentioned auth_request, someone mentioned XHR aka ajax requests. I think these are all separate issues from a possible google 3600 second thing.

For auth_request (which I'm now trying to use), the problem is that the request to /oauth2/auth gets a response with a Set-Cookie header, but that response is not sent to the user. The subsequent response from the actual application is. So the refreshed cookie cannot be set. I'm trying to convince nginx to get this header out of the auth_request response and put it in the actual response, but I have not been successful so far.

ploxiln avatar Apr 24 '17 21:04 ploxiln

This works for me, for refreshing the cookie via auth_request:

    location / {
        auth_request /oauth2/auth;
        error_page 401 = /oauth2/sign_in;
        auth_request_set $auth_cookie $upstream_http_set_cookie;

        proxy_pass http://apps;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Host $host;

        add_header Set-Cookie $auth_cookie;
    }

ploxiln avatar Apr 24 '17 21:04 ploxiln

@ploxiln It is multiple problems in one time. It is not working because google session expires in 1h if you are not active in 1h then after any javascript action, you get session refresh trying which remove session because it's expired and then trying to sign_in and no cookie set because you are not logged in. You need to refresh the whole page to sign_in and then you have next 1h of working and again if only javascript actions on your site (which is standard now) then you will get failed requests after 1h.

And yes someone mentioned XHR aka ajax requests this is a big problem because oauth2_proxy with google right now cannot be used without huge hacks with any javascript oriented site which is almost any site right now.

Log here with domains changed - i was back after night and need to refres whole page and sign in:

2017/05/18 06:32:12 oauthproxy.go:605: 127.0.0.1:41452 ("172.23.60.84") refreshing 10h7m48s old session cookie for Session{[email protected] token:true expires:2017-05-17 21:24:23 +0000 UTC} (refresh after 15m0s)
2017/05/18 06:32:12 oauthproxy.go:619: 127.0.0.1:41452 ("172.23.60.84") removing session. token expired Session{[email protected] token:true expires:2017-05-17 21:24:23 +0000 UTC}
172.23.60.84 - - [18/May/2017:06:32:12 +0000] my.example.com GET - "/oauth2/auth" HTTP/1.1 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" 401 21 0.000
172.23.60.84 - - [18/May/2017:06:32:12 +0000] my.example.com GET - "/oauth2/sign_in" HTTP/1.0 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" 200 2549 0.000
2017/05/18 06:32:15 oauthproxy.go:602: 127.0.0.1:41490 ("172.23.60.84") Cookie "_oauth_cookie_google_my-example.com" not present

After page refresh and sign in:

172.23.60.84 - - [18/May/2017:06:46:06 +0000] my.example.com GET - "/oauth2/start?rd=%2Fdatasources%3DAll" HTTP/1.0 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" 302 476 0.000
2017/05/18 06:46:07 oauthproxy.go:557: 127.0.0.1:45630 ("172.23.60.84") authentication complete Session{[email protected] token:true expires:2017-05-18 07:46:06 +0000 UTC}
172.23.60.84 - - [18/May/2017:06:46:07 +0000] my.example.com GET - "/oauth2/callback?state=pde4eacdkdsd9ds8s0962648155ds4e57:/datasources%3DAll&code=ds9k2d/P1sNtjC0dldld9s9snw8oxpSRbaP6qOP_fD342jj32" HTTP/1.0 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" 302 118 0.657
172.23.60.84 - [email protected] [18/May/2017:06:46:07 +0000] my.example.com GET - "/oauth2/auth" HTTP/1.1 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" 202 0 0.000
172.23.60.84 - [email protected] [18/May/2017:06:46:07 +0000] my.example.com GET - "/oauth2/auth" HTTP/1.1 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" 202 0 0.000
172.23.60.84 - [email protected] [18/May/2017:06:46:07 +0000] my.example.com GET - "/oauth2/auth" HTTP/1.1 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" 202 0 0.000
172.23.60.84 - [email protected] [18/May/2017:06:46:07 +0000] my.example.com GET - "/oauth2/auth" HTTP/1.1 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" 202 0 0.000

My oauth2 config:

http_address = "127.0.0.1:4185"
redirect_url = "https://my.example.com/oauth2/callback"
upstreams = [
     "http://127.0.0.1:54345/"
]
proxy_prefix = "/oauth2"
request_logging = true
pass_host_header = true
pass_user_headers = true
pass_access_token = true
email_domains = [
   "example.com"
]
client_id = "10290943283323-nasd1qikd32sdjdskmdpfodss9s6n.apps.googleusercontent.com"
client_secret = "7bDFMWKDSDFDSLM4542DVSS"
google_admin_email = "[email protected]
google_group = "[email protected]"
google_service_account_json = "/etc/oauth2-proxy-google.json"
login_url = "https://accounts.google.com/o/oauth2/v2/auth?access_type=offline"
cookie_name = "_oauth_cookie_google_my_example.com"
cookie_secret = "9N6T9fdfds0s9kdsfjdskf30"
cookie_domain = "my.exmple.com"
cookie_expire = "168h"
cookie_refresh = "15m"
cookie_secure = true
cookie_httponly = true
set_xauthrequest = true

And nginx conf section:

server {

    listen 443 spdy http2 ssl;
    server_name my.example.com;

    include conf.d/ssl_my_example_com;
    include conf.d/hsts.conf;

    access_log syslog:server=127.0.0.1,facility=local5,tag=access,severity=info custom_format;
    error_log syslog:server=127.0.0.1,facility=local5,tag=error,severity=error;

    location = /favicon.ico { access_log off; log_not_found off; }

    root   /var/www/html;
    index index.html index.php;

    location = /oauth2/auth {
      internal;
      proxy_pass http://oauth2proxy;
    }

    location /oauth2/ {
        proxy_pass http://oauth2proxy;
        proxy_set_header Host                    $host;
        proxy_set_header X-Real-IP               $remote_addr;
        proxy_set_header X-Scheme                $scheme;
        proxy_set_header X-Auth-Request-Redirect $request_uri;
    }

    location / {

      port_in_redirect off;

      include conf.d/cors.conf;

      auth_request /oauth2/auth;
      error_page 401 = /oauth2/sign_in;

      auth_request_set $user   $upstream_http_x_auth_request_user;
      auth_request_set $email  $upstream_http_x_auth_request_email;
      proxy_set_header X-Authenticated-User  $user;
      proxy_set_header X-Authenticated-Email $email;

      include conf.d/cors.conf;

      proxy_pass_request_headers on;
      proxy_set_header      X-Real-IP  $remote_addr;
      proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header      Host $http_host;
      proxy_redirect        off;
      proxy_connect_timeout 5s;

      auth_request_set $auth_cookie $upstream_http_set_cookie;
      add_header Set-Cookie $auth_cookie;

      proxy_pass http://go_backend;

    }

My workaround is to add dynamically to any site proxied through Nginx HTML code with an iframe which will be loaded in defined refresh rate and sign-in through normal GET if expires in the background, but this is a huge hack and not working with all possible scenarios. This needs separate configuration got sign_in to be automatic without a sign_in page in oauth2_proxy of course not as the Nginx config I added above.

szibis avatar May 18 '17 07:05 szibis

If those are your real cookie_secret and client_secret, it's probably better to not post those...

ploxiln avatar May 18 '17 14:05 ploxiln

Of course they are fake, randomly changed.

szibis avatar May 19 '17 04:05 szibis

@szibis @ploxiln What version of Oauth_proxy are you guys using? It works for me with OAuth2 Proxy version 2.0.1, not with the latest versions.

kishoreyellamraju avatar Jun 26 '17 21:06 kishoreyellamraju

I'm using oauth2_proxy 2.2.0 (and cookie refresh is still working for me, using github provider)

ploxiln avatar Jun 26 '17 21:06 ploxiln

@ploxiln I am having issues with Google login. I did few changes recently so I will give it a shot again.

The other issue that I see is with setting cookie_secure = true, when I set this config to true the OAuth session breaks in the new tab. when I access the same url in a new tab, it forces me to log in again.

kishoreyellamraju avatar Jun 26 '17 21:06 kishoreyellamraju

If cookie_secure has that effect, it could be because a plain http request is made by the browser, and reaches oauth2_proxy. If cookie_secure=true then that plain http request would not include the cookie, the request would be considered "not authenticated", and oauth2_proxy would respond with a cleared cookie to clean-up old state.

ploxiln avatar Jun 26 '17 22:06 ploxiln

This issue is quite old by now, but maybe my findings in https://github.com/bitly/oauth2_proxy/issues/632 help some others to fix their own problems. In my case I am not using Nginx but Traefik and proxying all traffic through the oauth proxy, not just the auth paths.

bcorijn avatar Jul 18 '18 14:07 bcorijn