Where can I "exchange code for token"

[rrnewton]

Google's OAuth docs show pictures of the back-and-forth protocol to get user consent, get a response code, and exchange it for a token that can actually be used for API calls.

(I've been doing this for a while with handa-gdata but without knowing how it works.)

Now I'm staring at the haddock Indices for gogol, gogol-core, and gogol-oauth2 and I see very little mention of "token"s. Where are the modules/types to:

• form the request URL for user consent?
• exchange the code for a token (type OAuthToken)?

[rrnewton]

FYI, this is something we do in Network.Google.OAuth2.exchangeCode in handa-gdata.

I was going to hack around this by making my own requests for auth. But I'm not sure how to reconcile that with newEnv. That is with the abstract view of credentials in the Credentials datatype, I can't see how to dig out the client-ID + client-secret to form the request URL's I'd need to.

[brendanhay]

Currently the generated gogol-* libraries have little way of supplanting a prospectively generated type with a reference to a type in gogol-core. For example, similarly in amazonka-* any Region type will be replaced with a reference to Network.AWS.Types.Region - the idea would be the same for Network.Google.Types.OAuthToken, but the generator extension to do this doesn't currently exist.

The second part to this is the currently supported Credentials mechanisms are not comprehensive - they're just what I personally use, with the code more or less translated from the google/oauth2client Python library. If there's a particular flow for say OAuth2 you'd find convenient, give me a rough idea what it might look like and I'll be happy to work on something for you to review. For example, there are currently ways to refresh tokens from metadata and accounts.google.com using the service_account or authorised_user types - I'd imagine there'd be a third mechanism supporting the OAuth2 flow/callback.

[brendanhay]

To slightly clarify my meandering first point above, you can import gogol-core.Network.Google.Types and use the OAuthToken { tokenToBS :: ByteString } type/constructor via Credentials.FromToken :: OAuthToken when calling newEnv, if you do the OAuth2 jig manually and construct your own token as a temporary workaround.

[rrnewton]

Thanks for the quick response. Let me clarify where I'm coming from and maybe there's a simple doc patch we can do in gogol or gogol-core that will/would-have pointed me in the right direction.

I'm quite ignorant about this stuff (web technology, web APIs, even how http works). But my hope was that Haskell libs like gogol could make it easy to use web APIs without understanding too much about them -- ideally by reading the type of the desired function and finding a way to satisfy it.

If we define "hello world" as calling some API function and accessing my own data (in a fusion table, or google doc or whatever), what then is the quickest path between from zero to hello world?

To respond specifically to your suggestion about the ByteString OAuthToken, I didn't know that we could represent everything we needed in one token string. In handa-gdata we cache a structure that looks like this:

OAuth2Tokens {accessToken = "...", refreshToken = "...", expiresIn = 3600 % 1, tokenType = "Bearer" }

So it has both the "access token" and "refresh token". Does the gogol-core OAuthToken correspond to one or both of these? All I know is that in the past, we did the "jig" and we got the access/refresh tokens, which together with client id/secret was enough to carry out future API calls without human intervention. I tried just providing an "accesstoken" to Credentials.FromToken but then where did it get the client ID/secret from? (And how could it ensure those matched? In my case newEnv errored out with an "Unauthorized" ServiceError).

Let me illustrate one way in which I find all this stuff not very discoverable and full of dead-ends. This is the first time I've ran into the option of "Application Default Credentials" with the files created by the gcloud CLI tool (which, incidentally refers to a client ID/secret that I can't find in my developer console). I found that gogol can find application_default_credentials.json and connect, and even run some API calls. But I haven't authorized it for "Fusion Tables", so when I try one of those calls it fails with "Forbidden". Ok, that makes sense, but at that point it's not clear from either the runtime error message, or the types of the Haskell API calls what to do next to fix this.

• Can I extend the authorization of the existing application_default_credentials by making a call somewhere and passing fusiontablesScope. Would that pop up a consent page? I don't see a call that accepts the OAuthScope in gogol, so I'm not sure what to do with values of that type.
• Can I manually log in somewhere in my browser to grant the authorization? (I can't even find the "project" in the developer console, so, not promising).

Maybe a "for newbies" tutorial would help here ;-).

[rrnewton]

P.S. I suppose a very strongly typed version of scopes would ensure that you'd authorized a scope in order for any API calls in that scope to type check. Would that ultimately be possible? I see someone has played with that idea in a different context here.

[brendanhay]

Thanks for spending the time to elaborate.

Regarding the points about documentation, hello world, examples etc. it's something I'm going to concentrate on - it's just unfortunate that the current stage is where early adopters may have to suffer through the discovery phase until I can complete some Haddock examples and so forth. Sorry. :) I hope to have this done reasonably soon though.

I'll leave my professional opinion about Google's documentation and Developer Console out of this, and just agree with you that some to-the-point documentation taking a user from credential creation through to sending requests is required, in all of the many possible credential permutations!

With the token refresh/secret - the Bearer type (isomorphic to your OAuth2Tokens type above) should be exposed, I'll tidy that up shortly.

The ability to supply scopes is actually something that came in with the fix for the other issue you raised in #6, constants are provided which document the available scopes. The API for this is less than ideal, but thankfully addressed by the next point: Scoping-the-scopes sounds great, there is in fact explicit metadata linking the operation and its required permissions here:

• https://github.com/brendanhay/gogol/blob/develop/gen/model/storage/v1/storage-api.json#L1171-L1175
• https://github.com/brendanhay/gogol/blob/baba589da006a93c7072804b18f7954e449a80d4/gogol-storage/gen/Network/Google/Storage/Types.hs#L351-L369

So it certainly seems possible, I think it'd be a nice addition overall. With regarding to vague 403 Forbidden error responses, that one seems a bit harder to manage - there is no information in the HTTP responses that you're not seeing in the error message, so the typed scopes currently seems the best way to mitigate seeing the errors in the first place.

[rrnewton]

Great, I will await those docs. In the meantime, I'm afraid I still didn't catch which permutation you use--what works for you the designer given the currently exposed interface?

[brendanhay]

I use two methods - the gcloud init created application_default_credentials.json when developing on my local machine, and once deployed to GCE instances the metadata endpoint available at http://169.254.169.254 is called to safely retrieve the credentials assigned to that specific instance.

The OAuth2 flow for what Google terms 'Installed Application Credentials' has just been added - primarily based off what google/oauth2client and handa-gdata do. (I hope that is OK.) I'm still working on this flow, but out of interest, how is this actually being used? Are you installing the handa-gdata CLI on development machines or actual production instances? How do you (if so) 'pop the browser/url' to obtain the authentication code in the latter case?

[rrnewton]

Yeah, so part of our impedance mismatch is that we're running on university machines and supercomputers, not on VMs that we manage (or on client devices like phones!). So it's sometimes hard to translate the docs to our use case ;-).

Basically what we use handa-gdata for is supporting the hsbencher library which runs benchmarks on Linux machines and then uploads the data to fusion tables. We launch jobs from Jenkins to run benchmarks on a group account and the group account caches the access/refresh token in its (NFS-mounted) home directory. I logged in once interactively to paste in the authorization token to get the access/refresh tokens, and they've been cached ever since.

So yes, the "production instancess" in this case are building Haskell source code (including our benchmarks) and installing handa-gdata.

It's good to know you use the application_default_credentials.json approach. I'm happy to deploy that file (with private key) to that group accounts home dir. (I just need to figure out how to authorize it with the extra scopes... which it sounds like you have some way of doing if you are able to make those default credentials usable.)

[brendanhay]

Just to clarify, the application_default_credentials.json is generated by gcloud init and is of type authorized_user. This type of credential has access the scopes of the project/application (aka what is enabled in the Developer Console).

The GOOGLE_APPLICATION_CREDENTIALS environment variable (documented here) defines a file path that will be read for either an authorized_user or service_account. It seems the latter could be suitable if you wanted to avoid the login/pasting the code (although it doesn't sound much of an issue). You can create a service account via:

Only the JSON format is supported, not P12.

To obtain what is ultimately inserted in the Authorization: Bearer ... header, the private key for the service account is used to sign a JWT with the specified scopes. These scopes are either explicitly passed to newEnv and used by the underlying Network.Google.Auth.discover function, or they can be passed manually using newEnvWith and one of the fromFile/fromFilePath etc. functions or FromAccount :: ServiceAccount -> [OAuthScope] -> Credentials constructor.

That said - I've started to add support for the Installed Application Flow which you can read the Haddock for here. The idea being you would use formURL to obtain the OAuthCode, and then construct Credentials to pass to newEnvWith via the FromClient :: OAuthClient -> OAuthCode -> Credentials constructor. The token exchange and any subsequent refreshes should occur how you expect, since what you have in handa-gdata is correct and any complexity on my end should hopefully only come from the need to support the multiple different authentication/authorization methods.

[kahlil29]

@brendanhay

To slightly clarify my meandering first point above, you can import gogol-core.Network.Google.Types and use the OAuthToken { tokenToBS :: ByteString } type/constructor via Credentials.FromToken :: OAuthToken when calling newEnv, if you do the OAuth2 jig manually and construct your own token as a temporary workaround.

Trying to do this but can't seem to find the Credentials.FromToken :: OAuthToken constructor for Credentials. Has it been removed? If yes, how can I go about the above flow you've suggested? I can get my OAuth Tokens and refresh them by myself, if necessary

[kahlil29]

To give some context, I'm trying to use OAuth2 to get consent from an end-user and then Create a Spreadsheet in the User's Drive and perform read/write operations on the Spreadsheet.

I've tried the following code using gogol-auth and gogol-sheets

getSheet :: Text -> Text -> IO Spreadsheet
getSheet sheetID authCode = do 
    logger <- newLogger Debug stdout
    mgr <- (newManager tlsManagerSettings)
    env <- newEnvWith (constructCredentials authCode) (logger)  mgr  <&> (envScopes .~ driveScope)
    runResourceT . runGoogle env $ send (spreadsheetsGet sheetID)

constructCredentials :: Text -> Credentials newCredentials takes the Auth Code (it has the ClientID and ClientSecret) and returns the Credentials to be used in the request.

The code compiles but when I run it, it gives me an error saying redirect_uri_mismatch. On further investigation I found that it sends a redirect_uri that is used by default for Installed Applications (Desktop/Android). Mine is a Web Application.

[Client Request] {
  host      = www.googleapis.com:443
  secure    = True
  method    = POST
  timeout   = ResponseTimeoutDefault
  redirects = 10
  path      = /oauth2/v4/token
  query     = 
  headers   = content-type: application/x-www-form-urlencoded
  body      = grant_type=authorization_code&client_id=<MyClientID>&client_secret=<MyClientSecret>&redirect_uri=urn:ietf:wg:oauth:2.0:oob
}
[Client Response] {
  status  = 400 Bad Request
  headers = vary: Origin; vary: X-Origin; content-type: application/json; charset=UTF-8; content-encoding: gzip; date: Sun, 26 Nov 2017 07:36:25 GMT; expires: Sun, 26 Nov 2017 07:36:25 GMT; cache-control: private, max-age=0; x-content-type-options: nosniff; x-frame-options: SAMEORIGIN; x-xss-protection: 1; mode=block; server: GSE; alt-svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"; transfer-encoding: chunked
}
[Refresh Error] Failure refreshing token from www.googleapis.com/oauth2/v4/token
*** Exception: TokenRefreshError (Status {statusCode = 400, statusMessage = "Bad Request"}) "redirect_uri_mismatch" (Just "Bad Request")

[klozovin]

@kahlil29 Did you by any chance figure it out? I'm trying to do something similar (but instead of Sheets I'm using YouTube) and getting the same error. It seems gogol makes an URL for "Installed Application" but not for a "Web server application". Any ideas how to that? Or is it something that gogol just can't do?

[kahlil29]

@kmelva I didn't figure out that part, sadly. Had to stick to using Network.OAuth.OAuth2.HttpClient for my OAuth2 authentication and then manually construct the API calls and send them using Network.Wreq.

However I did use gogol-sheets to construct my request bodies and to assign the incoming response bodies. It had nice lens accessors and objects which spared me some headaches :)

[klozovin]

@kahlil29 Thanks for the answer, I guess that's what I'll do also!

[NorfairKing]

To give some context, I'm trying to use OAuth2 to get consent from an end-user and then Create a Spreadsheet in the User's Drive and perform read/write operations on the Spreadsheet.

I've tried the following code using gogol-auth and gogol-sheets

getSheet :: Text -> Text -> IO Spreadsheet
getSheet sheetID authCode = do 
    logger <- newLogger Debug stdout
    mgr <- (newManager tlsManagerSettings)
    env <- newEnvWith (constructCredentials authCode) (logger)  mgr  <&> (envScopes .~ driveScope)
    runResourceT . runGoogle env $ send (spreadsheetsGet sheetID)

constructCredentials :: Text -> Credentials newCredentials takes the Auth Code (it has the ClientID and ClientSecret) and returns the Credentials to be used in the request.

The code compiles but when I run it, it gives me an error saying redirect_uri_mismatch. On further investigation I found that it sends a redirect_uri that is used by default for Installed Applications (Desktop/Android). Mine is a Web Application.

[Client Request] {
  host      = www.googleapis.com:443
  secure    = True
  method    = POST
  timeout   = ResponseTimeoutDefault
  redirects = 10
  path      = /oauth2/v4/token
  query     = 
  headers   = content-type: application/x-www-form-urlencoded
  body      = grant_type=authorization_code&client_id=<MyClientID>&client_secret=<MyClientSecret>&redirect_uri=urn:ietf:wg:oauth:2.0:oob
}
[Client Response] {
  status  = 400 Bad Request
  headers = vary: Origin; vary: X-Origin; content-type: application/json; charset=UTF-8; content-encoding: gzip; date: Sun, 26 Nov 2017 07:36:25 GMT; expires: Sun, 26 Nov 2017 07:36:25 GMT; cache-control: private, max-age=0; x-content-type-options: nosniff; x-frame-options: SAMEORIGIN; x-xss-protection: 1; mode=block; server: GSE; alt-svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"; transfer-encoding: chunked
}
[Refresh Error] Failure refreshing token from www.googleapis.com/oauth2/v4/token
*** Exception: TokenRefreshError (Status {statusCode = 400, statusMessage = "Bad Request"}) "redirect_uri_mismatch" (Just "Bad Request")

I'm having the exact same issue. It seems that this way of using gogol is just not supported yet.