gogol
gogol copied to clipboard
Where can I "exchange code for token"
Google's OAuth docs show pictures of the back-and-forth protocol to get user consent, get a response code, and exchange it for a token that can actually be used for API calls.
(I've been doing this for a while with handa-gdata but without knowing how it works.)
Now I'm staring at the haddock Indices for gogol
, gogol-core
, and gogol-oauth2
and I see very little mention of "token"s. Where are the modules/types to:
- form the request URL for user consent?
- exchange the code for a token (type
OAuthToken
)?
FYI, this is something we do in Network.Google.OAuth2.exchangeCode in handa-gdata.
I was going to hack around this by making my own requests for auth. But I'm not sure how to reconcile that with newEnv
. That is with the abstract view of credentials in the Credentials
datatype, I can't see how to dig out the client-ID + client-secret to form the request URL's I'd need to.
Currently the generated gogol-*
libraries have little way of supplanting a prospectively generated type with a reference to a type in gogol-core
. For example, similarly in amazonka-*
any Region
type will be replaced with a reference to Network.AWS.Types.Region
- the idea would be the same for Network.Google.Types.OAuthToken
, but the generator extension to do this doesn't currently exist.
The second part to this is the currently supported Credentials
mechanisms are not comprehensive - they're just what I personally use, with the code more or less translated from the google/oauth2client
Python library. If there's a particular flow for say OAuth2 you'd find convenient, give me a rough idea what it might look like and I'll be happy to work on something for you to review. For example, there are currently ways to refresh tokens from metadata and accounts.google.com
using the service_account
or authorised_user
types - I'd imagine there'd be a third mechanism supporting the OAuth2 flow/callback.
To slightly clarify my meandering first point above, you can import gogol-core.Network.Google.Types
and use the OAuthToken { tokenToBS :: ByteString }
type/constructor via Credentials.FromToken :: OAuthToken
when calling newEnv
, if you do the OAuth2 jig manually and construct your own token as a temporary workaround.
Thanks for the quick response. Let me clarify where I'm coming from and maybe there's a simple doc patch we can do in gogol
or gogol-core
that will/would-have pointed me in the right direction.
I'm quite ignorant about this stuff (web technology, web APIs, even how http works). But my hope was that Haskell libs like gogol could make it easy to use web APIs without understanding too much about them -- ideally by reading the type of the desired function and finding a way to satisfy it.
If we define "hello world" as calling some API function and accessing my own data (in a fusion table, or google doc or whatever), what then is the quickest path between from zero to hello world?
To respond specifically to your suggestion about the ByteString
OAuthToken, I didn't know that we could represent everything we needed in one token string. In handa-gdata
we cache a structure that looks like this:
OAuth2Tokens {accessToken = "...", refreshToken = "...", expiresIn = 3600 % 1, tokenType = "Bearer" }
So it has both the "access token" and "refresh token". Does the gogol-core OAuthToken
correspond to one or both of these? All I know is that in the past, we did the "jig" and we got the access/refresh tokens, which together with client id/secret was enough to carry out future API calls without human intervention. I tried just providing an "accesstoken" to Credentials.FromToken
but then where did it get the client ID/secret from? (And how could it ensure those matched? In my case newEnv
errored out with an "Unauthorized" ServiceError
).
Let me illustrate one way in which I find all this stuff not very discoverable and full of dead-ends. This is the first time I've ran into the option of "Application Default Credentials" with the files created by the gcloud
CLI tool (which, incidentally refers to a client ID/secret that I can't find in my developer console). I found that gogol can find application_default_credentials.json
and connect, and even run some API calls. But I haven't authorized it for "Fusion Tables", so when I try one of those calls it fails with "Forbidden"
. Ok, that makes sense, but at that point it's not clear from either the runtime error message, or the types of the Haskell API calls what to do next to fix this.
- Can I extend the authorization of the
existing application_default_credentials
by making a call somewhere and passingfusiontablesScope
. Would that pop up a consent page? I don't see a call that accepts theOAuthScope
ingogol
, so I'm not sure what to do with values of that type. - Can I manually log in somewhere in my browser to grant the authorization? (I can't even find the "project" in the developer console, so, not promising).
Maybe a "for newbies" tutorial would help here ;-).
P.S. I suppose a very strongly typed version of scopes would ensure that you'd authorized a scope in order for any API calls in that scope to type check. Would that ultimately be possible? I see someone has played with that idea in a different context here.
Thanks for spending the time to elaborate.
Regarding the points about documentation, hello world, examples etc. it's something I'm going to concentrate on - it's just unfortunate that the current stage is where early adopters may have to suffer through the discovery phase until I can complete some Haddock examples and so forth. Sorry. :) I hope to have this done reasonably soon though.
I'll leave my professional opinion about Google's documentation and Developer Console out of this, and just agree with you that some to-the-point documentation taking a user from credential creation through to sending requests is required, in all of the many possible credential permutations!
With the token refresh/secret - the Bearer
type (isomorphic to your OAuth2Tokens
type above) should be exposed, I'll tidy that up shortly.
The ability to supply scopes is actually something that came in with the fix for the other issue you raised in #6, constants are provided which document the available scopes. The API for this is less than ideal, but thankfully addressed by the next point: Scoping-the-scopes sounds great, there is in fact explicit metadata linking the operation and its required permissions here:
- https://github.com/brendanhay/gogol/blob/develop/gen/model/storage/v1/storage-api.json#L1171-L1175
- https://github.com/brendanhay/gogol/blob/baba589da006a93c7072804b18f7954e449a80d4/gogol-storage/gen/Network/Google/Storage/Types.hs#L351-L369
So it certainly seems possible, I think it'd be a nice addition overall. With regarding to vague 403 Forbidden
error responses, that one seems a bit harder to manage - there is no information in the HTTP responses that you're not seeing in the error message, so the typed scopes currently seems the best way to mitigate seeing the errors in the first place.
Great, I will await those docs. In the meantime, I'm afraid I still didn't catch which permutation you use--what works for you the designer given the currently exposed interface?
I use two methods - the gcloud init
created application_default_credentials.json
when developing on my local machine, and once deployed to GCE instances the metadata endpoint available at http://169.254.169.254
is called to safely retrieve the credentials assigned to that specific instance.
The OAuth2 flow for what Google terms 'Installed Application Credentials' has just been added - primarily based off what google/oauth2client
and handa-gdata
do. (I hope that is OK.) I'm still working on this flow, but out of interest, how is this actually being used? Are you installing the handa-gdata
CLI on development machines or actual production instances? How do you (if so) 'pop the browser/url' to obtain the authentication code in the latter case?
Yeah, so part of our impedance mismatch is that we're running on university machines and supercomputers, not on VMs that we manage (or on client devices like phones!). So it's sometimes hard to translate the docs to our use case ;-).
Basically what we use handa-gdata for is supporting the hsbencher
library which runs benchmarks on Linux machines and then uploads the data to fusion tables. We launch jobs from Jenkins to run benchmarks on a group account and the group account caches the access/refresh token in its (NFS-mounted) home directory. I logged in once interactively to paste in the authorization token to get the access/refresh tokens, and they've been cached ever since.
So yes, the "production instancess" in this case are building Haskell source code (including our benchmarks) and installing handa-gdata.
It's good to know you use the application_default_credentials.json
approach. I'm happy to deploy that file (with private key) to that group accounts home dir. (I just need to figure out how to authorize it with the extra scopes... which it sounds like you have some way of doing if you are able to make those default credentials usable.)
Just to clarify, the application_default_credentials.json
is generated by gcloud init
and is of type authorized_user
. This type of credential has access the scopes of the project/application (aka what is enabled in the Developer Console).
The GOOGLE_APPLICATION_CREDENTIALS
environment variable (documented here) defines a file path that will be read for either an authorized_user
or service_account
. It seems the latter could be suitable if you wanted to avoid the login/pasting the code (although it doesn't sound much of an issue). You can create a service account via:
Only the JSON format is supported, not P12.
To obtain what is ultimately inserted in the Authorization: Bearer ...
header, the private key for the service account is used to sign a JWT with the specified scopes. These scopes are either explicitly passed to newEnv
and used by the underlying Network.Google.Auth.discover
function, or they can be passed manually using newEnvWith
and one of the fromFile/fromFilePath
etc. functions or FromAccount :: ServiceAccount -> [OAuthScope] -> Credentials
constructor.
That said - I've started to add support for the Installed Application Flow which you can read the Haddock for here. The idea being you would use formURL
to obtain the OAuthCode
, and then construct Credentials
to pass to newEnvWith
via the FromClient :: OAuthClient -> OAuthCode -> Credentials
constructor. The token exchange and any subsequent refreshes should occur how you expect, since what you have in handa-gdata
is correct and any complexity on my end should hopefully only come from the need to support the multiple different authentication/authorization methods.
@brendanhay
To slightly clarify my meandering first point above, you can import gogol-core.Network.Google.Types and use the OAuthToken { tokenToBS :: ByteString } type/constructor via Credentials.FromToken :: OAuthToken when calling newEnv, if you do the OAuth2 jig manually and construct your own token as a temporary workaround.
Trying to do this but can't seem to find the Credentials.FromToken :: OAuthToken
constructor for Credentials. Has it been removed?
If yes, how can I go about the above flow you've suggested? I can get my OAuth Tokens and refresh them by myself, if necessary
To give some context, I'm trying to use OAuth2 to get consent from an end-user and then Create a Spreadsheet in the User's Drive and perform read/write operations on the Spreadsheet.
I've tried the following code using gogol-auth
and gogol-sheets
getSheet :: Text -> Text -> IO Spreadsheet
getSheet sheetID authCode = do
logger <- newLogger Debug stdout
mgr <- (newManager tlsManagerSettings)
env <- newEnvWith (constructCredentials authCode) (logger) mgr <&> (envScopes .~ driveScope)
runResourceT . runGoogle env $ send (spreadsheetsGet sheetID)
constructCredentials :: Text -> Credentials newCredentials
takes the Auth Code (it has the ClientID
and ClientSecret
) and returns the Credentials
to be used in the request.
The code compiles but when I run it, it gives me an error saying redirect_uri_mismatch
. On further investigation I found that it sends a redirect_uri
that is used by default for Installed Applications
(Desktop/Android). Mine is a Web Application.
[Client Request] {
host = www.googleapis.com:443
secure = True
method = POST
timeout = ResponseTimeoutDefault
redirects = 10
path = /oauth2/v4/token
query =
headers = content-type: application/x-www-form-urlencoded
body = grant_type=authorization_code&client_id=<MyClientID>&client_secret=<MyClientSecret>&redirect_uri=urn:ietf:wg:oauth:2.0:oob
}
[Client Response] {
status = 400 Bad Request
headers = vary: Origin; vary: X-Origin; content-type: application/json; charset=UTF-8; content-encoding: gzip; date: Sun, 26 Nov 2017 07:36:25 GMT; expires: Sun, 26 Nov 2017 07:36:25 GMT; cache-control: private, max-age=0; x-content-type-options: nosniff; x-frame-options: SAMEORIGIN; x-xss-protection: 1; mode=block; server: GSE; alt-svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"; transfer-encoding: chunked
}
[Refresh Error] Failure refreshing token from www.googleapis.com/oauth2/v4/token
*** Exception: TokenRefreshError (Status {statusCode = 400, statusMessage = "Bad Request"}) "redirect_uri_mismatch" (Just "Bad Request")
@kahlil29 Did you by any chance figure it out? I'm trying to do something similar (but instead of Sheets I'm using YouTube) and getting the same error. It seems gogol
makes an URL for "Installed Application" but not for a "Web server application". Any ideas how to that? Or is it something that gogol just can't do?
@kmelva I didn't figure out that part, sadly. Had to stick to using Network.OAuth.OAuth2.HttpClient for my OAuth2 authentication and then manually construct the API calls and send them using Network.Wreq.
However I did use gogol-sheets
to construct my request bodies and to assign the incoming response bodies. It had nice lens accessors and objects which spared me some headaches :)
@kahlil29 Thanks for the answer, I guess that's what I'll do also!
To give some context, I'm trying to use OAuth2 to get consent from an end-user and then Create a Spreadsheet in the User's Drive and perform read/write operations on the Spreadsheet.
I've tried the following code using
gogol-auth
andgogol-sheets
getSheet :: Text -> Text -> IO Spreadsheet getSheet sheetID authCode = do logger <- newLogger Debug stdout mgr <- (newManager tlsManagerSettings) env <- newEnvWith (constructCredentials authCode) (logger) mgr <&> (envScopes .~ driveScope) runResourceT . runGoogle env $ send (spreadsheetsGet sheetID)
constructCredentials :: Text -> Credentials newCredentials
takes the Auth Code (it has theClientID
andClientSecret
) and returns theCredentials
to be used in the request.The code compiles but when I run it, it gives me an error saying
redirect_uri_mismatch
. On further investigation I found that it sends aredirect_uri
that is used by default forInstalled Applications
(Desktop/Android). Mine is a Web Application.[Client Request] { host = www.googleapis.com:443 secure = True method = POST timeout = ResponseTimeoutDefault redirects = 10 path = /oauth2/v4/token query = headers = content-type: application/x-www-form-urlencoded body = grant_type=authorization_code&client_id=<MyClientID>&client_secret=<MyClientSecret>&redirect_uri=urn:ietf:wg:oauth:2.0:oob } [Client Response] { status = 400 Bad Request headers = vary: Origin; vary: X-Origin; content-type: application/json; charset=UTF-8; content-encoding: gzip; date: Sun, 26 Nov 2017 07:36:25 GMT; expires: Sun, 26 Nov 2017 07:36:25 GMT; cache-control: private, max-age=0; x-content-type-options: nosniff; x-frame-options: SAMEORIGIN; x-xss-protection: 1; mode=block; server: GSE; alt-svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"; transfer-encoding: chunked } [Refresh Error] Failure refreshing token from www.googleapis.com/oauth2/v4/token *** Exception: TokenRefreshError (Status {statusCode = 400, statusMessage = "Bad Request"}) "redirect_uri_mismatch" (Just "Bad Request")
I'm having the exact same issue. It seems that this way of using gogol
is just not supported yet.