Cache token / transport confusion

[billmccord]

Hi, I'm having a really hard time figuring out how to accomplish what I want to do. Essentially, I would like to cache the access and refresh tokens so that I don't have to ask the user to authenticate every time I want to create a new client. It seems like the Transport may have been added for this purpose, but an example would make things much clearer. Thanks for the awesome library and any help you can provide!

[billmccord]

After digging into the code a bit more, it seems like perhaps when I'm finished using an OAuth2 Client and want to cache the token away for later reuse I could:

1. Get the Transport from the http.Client
2. Cast it to an oauth2.Transport
3. Get the TokenSource from the oauth2.Transport
4. Get the latest Token from the TokenSource
5. Save the Token Is this a recommended approach or is there an easier way to do this by injecting some kind of callback function that gets notified when a Token is refreshed?

[jfcote87]

You could also just cache the Token from created from the Config.Exchange() func so that you only cache Token once because the refresh token never expires.

[billmccord]

Every time I refresh tokens on the OAuth library I'm using I get a new refresh token. Is that not standard? On Jan 31, 2015 1:48 AM, "Jim Cote" [email protected] wrote:

You could also just cache the Token from created from the Config.Exchange() func so that you only cache Token once because the refresh token never expires.

— Reply to this email directly or view it on GitHub https://github.com/golang/oauth2/issues/84#issuecomment-72240828.

[jfcote87]

Token.RefreshToken stays the same. Token.AccessToken changes.

See if the following gist explains it any better.

https://gist.github.com/jfcote87/89eca3032cd5f9705ba3

[billmccord]

Jim, thanks for posting the gist. That was helpful, but unless I'm misunderstanding something I think there are still cases where the token could be lost. Consider the following and let me know if this is a possible scenario:

1. I retrieve the token from my datastore and, while it is valid when I check it, it only has a few milliseconds left before expiration.
2. I create the new Client with the token.
3. As I'm using the Client, the token expires.
4. The RoundTrip method on the Client's Transport attempt to get the Token.
5. The reuseTokenSource determines that the token is invalid.
6. Now, the tokenRefresher's Token method is invoked.
7. retrieveToken is called and fetches a new token.
8. The Token is completely replaced (new AccessToken, RefreshToken, and Expiry.)

In this scenario, I would have no way of knowing the token was modified unless I followed my process above of recaching the token or, at the very least, verifying that the token didn't change while I was using the Client. This is because the Client automatically refreshes the Token for you as a convenience and, unfortunately, doesn't tell you that it did it via a callback.

Does that make sense or am I still missing something?

[jfcote87]

The RefreshToken does not change. so you don't have to worry about the background refresh. The tokenRefresher struct stores the RefreshToken in the oldToken field and this field is not updated again.
https://github.com/golang/oauth2/blob/master/oauth2.go#L221-L227

*note that the RefreshToken field of the returned Token pointer is updated by the passed refresh token. The tokenRefresher is not updated by the returned Token. https://github.com/golang/oauth2/blob/master/oauth2.go#L345-L347

[billmccord]

Jim, unfortunately, this does not seem to be the case. I built a test based on the tests included with oauth2 here: https://gist.github.com/billmccord/4247b0c4d2a6b5a4d09f

You can see from my gist that if the token expires after being used to create an oauth2.Client, then another request will be made in the background to obtain a new token (new access_token and new refresh_token) before proceeding with the request.

Unfortunately, at this point the refresh_token is lost because the ORIGINAL_REFRESH_TOKEN is no longer valid after the NEW_REFRESH_TOKEN is generated according to the OAuth2 spec.

Therefore, it is possible that the token that I stored per your suggestion will become invalid and the only way I can see to ensure that I have the right tokens is to always get the Token again from the Client after use and ensure that a) it hasn't changed from what I have stored or b) update what I have stored with the changed Token I got from the Client.

[jfcote87]

I think I see where we might be talking past each other and that an actual bug exists. I tested code using Google's client apis and their implementation of oauth2. According to Google's oauth documentation , refresh tokens do not expire but must be revoked. In my tests, a new refresh token is only generated during a code exchange (i.e. Config.Exchange() ). In a token refresh call (tokenRefresher.Token() ) the refresh_token field is omitted/left blank. This is allowed in the oauth2 spec as sending a new refresh_token during a refresh operation is optional (oauth2 doc section 1.5), not required.

You have found a bug. If a new refresh_token is generated during a refresh, the token source refresh token is not updated. See test code for an example.

Back to your original question of how to cache a Token each time it is refreshed. A hook would need to be added to Config and ReuseTokenSource().

[billmccord]

Thanks for validating this. The OAuth provider I'm calling uses a library that always generates a new refresh_token and there isn't an option to not do this yet. Since it is optional, I agree, that a hook would be necessary to get the updated Token in cases where it is renewed. Since that hook doesn't exist yet, I suppose that my workaround of getting the Token from the client when I'm finished using it is probably the only option?

e.g. // --- Obtain token from storage. --- // --- Create and use client with stored token. --- // Get latest token from client. newToken, err := client.Transport.(*oauth2.Transport).Source.Token() // --- Store revised token (if changed.) ---

It isn't pretty, but it seems like a reasonable workaround for now.

[rakyll]

Does the provider return a different refresh token each time you ask for a new access token?

[billmccord]

Yes. You can see here that issue_refresh_token is hard-coded to true for the refresh_token grant_type: https://github.com/FriendsOfSymfony/oauth2-php/blob/b0e57e17c84175a51af01cef7bbb2961261c84ad/lib/OAuth2.php#L840-L842

[rakyll]

Their API is not complaint with the OAuth 2.0 spec, and we have no intention to support provider-specific non-spec features.

[billmccord]

Two points:

1. I'm not clear on how it isn't compliant with the OAuth 2.0 spec when the spec specifically says in section 1.5 (emphasis, mine): " (H) The authorization server authenticates the client and validates the refresh token, and if valid issues a new access token (and optionally, a new refresh token)."
2. Haven't you already? https://github.com/golang/oauth2/blob/master/oauth2.go#L387-L395

[jfcote87]

https://tools.ietf.org/html/rfc6749#section-6 Section 6. Refreshing an Access Token (last paragraph, emphasis in original)

The authorization server MAY issue a new refresh token, in which case the client MUST discard the old refresh token and replace it with the new refresh token. The authorization server MAY revoke the old refresh token after issuing a new refresh token to the client. If a new refresh token is issued, the refresh token scope MUST be identical to that of the refresh token included by the client in the request.

https://tools.ietf.org/html/rfc6749#section-10.4 Section 10.4. Refresh Tokens 4th paragraph

For example, the authorization server could employ refresh token rotation in which a new refresh token is issued with every access token refresh response. The previous refresh token is invalidated but retained by the authorization server. If a refresh token is compromised and subsequently used by both the attacker and the legitimate client, one of them will present an invalidated refresh token, which will inform the authorization server of the breach.

[billmccord]

Jim, thanks, that seems to validate that the bug lies with this project because it doesn't provide a mechanism for obtaining the new refresh_token in situations where the refresh_token is replaced during the Refreshing an Access Token operation, correct?

If so, I can file a close this out and create a more specific issue OR we can just use this to track.

[jfcote87]

Bill, I have a CL ready that fixes the refresh token problem and will upload later today.

@rakyll are you ok with calling this a bug? Should we start a new issue to discuss caching difficulties?

[adg]

@jfcote87 It looks like a bug to me. I think we should start a new issue to discuss caching strategies.

[mgenov]

@adg is there open issue about caching strategies?

[adg]

Not that I know of.

On 4 May 2015 at 04:53, Miroslav Genov [email protected] wrote:

@adg https://github.com/adg is there open issue about caching strategies?

— Reply to this email directly or view it on GitHub https://github.com/golang/oauth2/issues/84#issuecomment-98522728.

[johnl]

For reference, there used to be support for token caching but it was removed for some reason: https://github.com/golang/oauth2/commit/93ad3f4a9ef21ece608bdf66c177b7573de9fcf7

[abourget]

So is this fixed, then ?

[rakyll]

@johnl, the trivial cache implementation is removed because we were not able to come up with a generic and useful cacher interface. I think a wrapper cacher RoundTripper is the perfect solution although it is not documented anywhere.

[gebv]

If the token has been restored, i can access a resource. But token is not auto-refreshed. Why?

sourceToken := oauth2.ReuseTokenSource(nil, &fileTokenSource{accessToken, refreshToken, expiry})

client := &http.Client{
    Transport: &oauth2.Transport{
        Base:   ContextTransport(oauth2.NoContext), // from internal/transport.go
        Source: oauth2.ReuseTokenSource(nil, sourceToken),
    },
}

client.Get("...")
// access the resource
// long time
// token expired
client.Get("...")
// and not auto-refreshed

[gebv]

When your mind is fresh... no problems Working code

sourceToken := oauth2.ReuseTokenSource(nil, &fileTokenSource{accessToken, refreshToken, expiry})
t, _ := sourceToken.Token()
client = conf.Client(oauth2.NoContext, t)
client.Get("...")
// token is auto-updated

[lewgun]

@rakyll I think a wrapper cacher RoundTripper is the perfect solution although it is not documented anywhere. is it here now ?

[arvenil]

Is there currently any workaround for this problem?

[arvenil]

Ok, I see bug with refreshing token got fixed: https://github.com/golang/oauth2/commit/cc2494a288f7645968af9c7293faf02e6371a377 right?

However, original question remains - what's currently proper way to save and restore token? Is there any kind of event when token get's changed? Any example how to achieve that?

[arvenil]

Below pseudo code seems to work. Is this a way to go?

    config := &oauth2.Config{
        ClientID: "App",
        ClientSecret: "It's not a secret",
        Endpoint: oauth2.Endpoint{
            TokenURL: apiUrl + "/token",
        },
    }
    restoredToken := &oauth2.Token{
        AccessToken: savedToken.AccessToken,
        RefreshToken: savedToken.RefreshToken,
        Expiry: savedToken.Expiry,
        TokenType: savedToken.TokenType,
    }
    tokenSource := config.TokenSource(oauth2.NoContext, restoredToken)
    client := oauth2.NewClient(oauth2.NoContext, tokenSource)
    savedToken, err = tokenSource.Token()
    if err != nil {
        return err
    }

    r, err  := client.Get("...")
    if err != nil {
        return err
    }

[rakyll]

@arvenil, a wrapper RoundTripper would look like what's below:

type cacherTransport struct {
    Base *oauth2.Transport
}

func (c *cacherTransport) RoundTrip(req *http.Request) (resp *http.Response, err error) {
    tok, err := c.Base.Source.Token()
    if err != nil {
        return nil, err
    }
    resp, err = c.Base.RoundTrip(req)
    if err != nil {
        return nil, err
    }
    newTok, err := c.Base.Source.Token()
    if err != nil {
        // TODO: handle error
    }
    if tok.AccessToken != newTok.AccessToken {
        // TODO: cache the refreshed token
    }
    return resp, nil
}

func Example_tokenCache() {
    conf := &oauth2.Config{
        ClientID:     "YOUR_CLIENT_ID",
        ClientSecret: "YOUR_CLIENT_SECRET",
        Scopes:       []string{"SCOPE1", "SCOPE2"},
        Endpoint: oauth2.Endpoint{
            AuthURL:  "https://provider.com/o/oauth2/auth",
            TokenURL: "https://provider.com/o/oauth2/token",
        },
    }

    var tok *oauth2.Token
    ts := conf.TokenSource(oauth2.NoContext, tok)
    tr := &oauth2.Transport{Source: ts}

    client := &http.Client{
        Transport: &cacherTransport{Base: tr},
    }
    client.Get("...")
}

[arvenil]

Thank you very much @rakyll Before your example I've tried different approach and ended up with wrapper around TokenSource and that seems to work too.

I wonder if there is any advantage or disadvantage of any of those solutions - in any case if someone wants to experiment here goes the code:)

type TokenSaver interface {
    Save(*oauth2.Token) error
}

func newCachedTokenSource(src oauth2.TokenSource, ts TokenSaver) oauth2.TokenSource {
    return &cachedTokenSource{
        pts: src,
        ts:  ts,
    }
}

type cachedTokenSource struct {
    pts oauth2.TokenSource // called when t is expired.
    ts  TokenSaver
}

func (s *cachedTokenSource) Token() (*oauth2.Token, error) {
    t, err := s.pts.Token()
    if err != nil {
        return nil, err
    }
    if err := s.ts.Save(t); err != nil {
        return nil, err
    }
    return t, nil
}

// example stolen from @rakyll ;)
func Example_tokenCache() {
    conf := &oauth2.Config{
        ClientID:     "YOUR_CLIENT_ID",
        ClientSecret: "YOUR_CLIENT_SECRET",
        Scopes:       []string{"SCOPE1", "SCOPE2"},
        Endpoint: oauth2.Endpoint{
            AuthURL:  "https://provider.com/o/oauth2/auth",
            TokenURL: "https://provider.com/o/oauth2/token",
        },
    }

    var tok *oauth2.Token // nil or restored token
    ts := conf.TokenSource(oauth2.NoContext, tok)
    tokenSaver := newTokenSaver() // here goes actuall implementation of TokenSaver interface
    cts := newCachedTokenSource(ts, tokenSaver)
    client := oauth2.NewClient(oauth2.NoContext, cachedTokenSource)
    client.Get("...")
}