FromFile Credentials not working with service_account type

[rrnewton]

gogol is fantastically exciting! (But trying to stumble through OAuth is painful... can the core gogol lib walk us through this a bit more for newbies?)

Here's my current stumbling block. Currently the docs on the Credentials data type say that its ok to use "service_account" credentials:

Load the Application Default Credentials from a specific file path. The file can be formatted as either a service_account or an authorized_user.

Ok, so I followed Google's instructions for "Application Default Credentials", and got a json file with a private key in it, which says it is in fact of type "service_account". Specifically, it's of the form:

{
  "private_key_id": "...",
  "private_key": "...",
  "client_email": "[email protected]",
  "client_id": "....apps.googleusercontent.com",
  "type": "service_account"
}

Then for my Credentials data type I point it at this file with FromFile "blah.json". But when trying to execute newEnv I get this error:

mytest-exe: InvalidFileError "blah.json" "key \"client_secret\" not present"

Hmm, that looks like it's trying to use the other form of credentials, not the "service_account" type...

CC @craigcitro

[rrnewton]

Reading through the code a bit, and it looks like this json file should match the FromJSON instance for ServiceAccount. And yet we seem to be getting an error from the AuthorisedUser parse. It's almost as though the disjunction isn't working right here:

instance FromJSON Parse where
    parseJSON o = SA <$> parseJSON o <|> AU <$> parseJSON o

[brendanhay]

A hesitant guess here is the private certificate in "private_key" for your ServiceAccount is failing to parse, but due to Parser instances and non-cumulative/monoidal errors it's simply showing the latter failure.

I'll generate some more personal ServiceAccount credentials and see if I can recreate to verify the above guess.

But additionally here are two possible improvements:

• Attempt to parse each inhabitant of the sum separately (no disjunction via <|>) and then sensibly combine the errors if neither parse succeeds.
• Remove FromFile in favour of using FromSerivceAccountFile and FromAuthorisedUserFile, or something similarly explicit.

[brendanhay]

I've pushed some changes to develop containing fixes for the above. The caveat is that now the newEnv requires a list of OAuth scopes to be supplied before the initial token refresh happens as out of the current credential mechanisms only the service_account method actually requires these.

Overall this feels less than ideal - since it requires the user to pass in scopes pre-initialisation / runGoogle, and if a subset of authorisation scopes is required, you'd effectively have to reinitialise, which seems like it'd encourage just using the broadest scope possible.

[rrnewton]

Ah, ok, I checked out the new type signature on the develop branch. But I'm afraid I don't get what it does, the current haddock says:

-- | Creates a new environment with a new 'Manager' without debug logging
-- and uses 'getAuth' to expand/discover the supplied 'Credentials'.
-- Lenses from 'HasEnv' can be used to further configure the resulting 'Env'.
--
-- /See:/ 'newEnvWith', 'defaultScopes'.

Is the semantics to ensure access to that list of scopes before running the supplied computation? But if the scopes are not available, this function won't do anything to try to get them authorized, will it?

[brendanhay]

Correct, it does not attempt to perform the full OAuth flow to request access to scopes. The scopes are used by credentials of the format:

{
  "private_key_id": "3703ad77e5ece90DFa",
  "private_key": "-----BEGIN PRIVATE KEY-----\n...\n",
  "client_email": "[email protected]",
  "client_id": "...usercontent.com",
  "type": "service_account"
}

To use credentials of the above style, the access token is retrieved by signing and authorising a JSON Web Token (JWT) with the scopes that are supplied to newEnv. If you supply incorrect scopes or an empty scope list obtaining the initial access token will fail with everyone's favourite 403 Forbidden and a token refresh error. This style of credentials is "grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer".

The other credential mechanisms such as the application_default_credentials.json generated by gcloud init, retrieving local compute instance metadata from http://169.254.169.254, and the soon to be exposed Bearer (access_token, refresh_token, expiry) type all have implicit scopes. These are "grant_type": "refresh_token".

The API for this, supplying scopes, should change or fade away for the cases it is unnecessary (It should probably be packed into the Credentials constructor now, not exposed via newEnv). The grant/revoke OAuth flow (I believe it is termed "grant_type": "client_credentails") is unimplemented. It sounds like this is the main way you want to provide credentials, so I'll focus on covering it.