git-ecosystem
Use distinct host for Bitbucket refresh token.
BitbucketHostProvider cleverly stores an OAuth refresh token as an independent credential with a variation on the path.
https://github.com/GitCredentialManager/git-credential-manager/blob/569d10b9eca391b3047af910037a89e62ead7cfb/src/shared/Atlassian.Bitbucket/BitbucketHostProvider.cs#L460-L466
However when using store credential.credentialstore=cache sometimes BitbucketHostProvider queries the store for the actual credential and returns the refresh token by mistake, causing authentication to fail. This happens because gitcredential helpers https://git-scm.com/docs/gitcredentials such as https://git-scm.com/docs/git-credential-cache match on host ignoring path.
By default, Git does not consider the "path" component of an http URL to be worth matching via external helpers. This means that a credential stored for https://example.com/foo.git will also be used for https://example.com/bar.git.
A simple fix would be to store the refresh token under a distinct host.
Example trace below:
- First push: No credential is found, a credential is generated, and the refresh token is stored.
- Second push expected: no credential is retrieved. The refresh token is retrieved, and used to generate a new credential.
- Second push what actually happens: a credential is retrieved (actually the refresh token). The retrieved credential is tested and rejected. The refresh token is retrieved, and used to generate a new credential.
This changes avoids the unexpected steps in bold.
me@penguin ~/cef-project (master)> git credential-cache exit me@penguin ~/cef-project (master)> git push 09:49:44.255135 ...re/Application.cs:95 trace: [RunInternalAsync] Version: 2.0.664.60794 09:49:44.269003 ...re/Application.cs:96 trace: [RunInternalAsync] Runtime: .NET 5.0.13 09:49:44.269052 ...re/Application.cs:97 trace: [RunInternalAsync] Platform: Linux (x86-64) 09:49:44.269065 ...re/Application.cs:98 trace: [RunInternalAsync] OSVersion: Linux penguin 5.4.157-17185-gbd27b903c738 #1 SMP PREEMPT Sat Jan 1 19:28:03 PST 2022 x86_64 GNU/Linux 09:49:44.269191 ...re/Application.cs:99 trace: [RunInternalAsync] AppPath: /home/me/git-credential-manager/out/shared/Git-Credential-Manager/bin/Debug/net5.0/git-credential-manager-core 09:49:44.269240 ...e/Application.cs:100 trace: [RunInternalAsync] Arguments: get 09:49:44.323010 ...GitCommandBase.cs:33 trace: [ExecuteAsync] Start 'get' command... 09:49:44.330633 ...GitCommandBase.cs:47 trace: [ExecuteAsync] Detecting host provider for input: 09:49:44.331348 ...GitCommandBase.cs:48 trace: [ExecuteAsync] protocol=https 09:49:44.331387 ...GitCommandBase.cs:48 trace: [ExecuteAsync] host=bitbucket.org 09:49:44.331401 ...GitCommandBase.cs:48 trace: [ExecuteAsync] username=hickford 09:49:44.363525 ...viderRegistry.cs:149 trace: [GetProviderAsync] Performing auto-detection of host provider. 09:49:44.367469 ...viderRegistry.cs:158 trace: [GetProviderAsync] Auto-detect probe timeout is 2 ms. 09:49:44.369600 ...viderRegistry.cs:166 trace: [GetProviderAsync] Checking against 3 host providers registered with priority 'Normal'. 09:49:44.370791 ...GitCommandBase.cs:50 trace: [ExecuteAsync] Host provider 'Bitbucket' was selected. 09:49:44.375620 ...tHostProvider.cs:307 trace: [GetSupportedAuthenticationModes] https://bitbucket.org/ is bitbucket.org - authentication schemes: 'All' 09:49:44.379774 ...tHostProvider.cs:103 trace: [GetStoredCredentials] Look for existing credentials under https://bitbucket.org ... 09:49:44.395503 ...tHostProvider.cs:109 trace: [GetStoredCredentials] No stored credentials found 09:49:44.397122 ...tHostProvider.cs:126 trace: [GetRefreshedCredentials] Refresh credentials... 09:49:44.399006 ...tHostProvider.cs:131 trace: [GetRefreshedCredentials] Checking for refresh token... 09:49:44.400920 ...tHostProvider.cs:138 trace: [GetRefreshedCredentials] No stored refresh token found 09:49:44.401081 ...tHostProvider.cs:146 trace: [GetRefreshedCredentials] Prompt for credentials... Select an authentication method for 'https://bitbucket.org/': 1. OAuth (default) 2. Username/password option (enter for default): 09:49:44.418076 .../LinuxTerminal.cs:45 trace: [.ctor] Setting terminal echo state to 'True' 1 09:49:46.194738 ...tHostProvider.cs:254 trace: [GetOAuthCredentialsViaInteractiveBrowserFlow] Starting OAuth authentication flow... 09:49:46.202425 ...pClientFactory.cs:58 trace: [CreateClient] Creating new HTTP client instance... 09:49:47.708171 ...tHostProvider.cs:258 trace: [GetOAuthCredentialsViaInteractiveBrowserFlow] Resolving username for OAuth credential... 09:49:47.710852 ...tbucketRestApi.cs:74 trace: [GetUserInformationAsync] HTTP: GET https://api.bitbucket.org/2.0/user 09:49:47.711094 ...pClientFactory.cs:58 trace: [CreateClient] Creating new HTTP client instance... 09:49:48.193313 ...tbucketRestApi.cs:77 trace: [GetUserInformationAsync] HTTP: Response 200 [OK] 09:49:48.210357 ...tHostProvider.cs:260 trace: [GetOAuthCredentialsViaInteractiveBrowserFlow] Username for OAuth credential is 'hickford' 09:49:48.210416 ...tHostProvider.cs:263 trace: [GetOAuthCredentialsViaInteractiveBrowserFlow] Storing new refresh token... 09:49:48.214356 ...tHostProvider.cs:265 trace: [GetOAuthCredentialsViaInteractiveBrowserFlow] Refresh token was successfully stored. 09:49:48.215681 ...GitCommandBase.cs:54 trace: [ExecuteAsync] End 'get' command... remote: Forbidden fatal: unable to access 'https://bitbucket.org/chromiumembedded/cef-project.git/': The requested URL returned error: 403 me@penguin ~/cef-project (master) [128]> git push 09:49:50.201027 ...re/Application.cs:95 trace: [RunInternalAsync] Version: 2.0.664.60794 09:49:50.214549 ...re/Application.cs:96 trace: [RunInternalAsync] Runtime: .NET 5.0.13 09:49:50.214594 ...re/Application.cs:97 trace: [RunInternalAsync] Platform: Linux (x86-64) 09:49:50.214606 ...re/Application.cs:98 trace: [RunInternalAsync] OSVersion: Linux penguin 5.4.157-17185-gbd27b903c738 #1 SMP PREEMPT Sat Jan 1 19:28:03 PST 2022 x86_64 GNU/Linux 09:49:50.214738 ...re/Application.cs:99 trace: [RunInternalAsync] AppPath: /home/me/git-credential-manager/out/shared/Git-Credential-Manager/bin/Debug/net5.0/git-credential-manager-core 09:49:50.214790 ...e/Application.cs:100 trace: [RunInternalAsync] Arguments: get 09:49:50.271586 ...GitCommandBase.cs:33 trace: [ExecuteAsync] Start 'get' command... 09:49:50.278756 ...GitCommandBase.cs:47 trace: [ExecuteAsync] Detecting host provider for input: 09:49:50.279469 ...GitCommandBase.cs:48 trace: [ExecuteAsync] protocol=https 09:49:50.279512 ...GitCommandBase.cs:48 trace: [ExecuteAsync] host=bitbucket.org 09:49:50.279526 ...GitCommandBase.cs:48 trace: [ExecuteAsync] username=hickford 09:49:50.314196 ...viderRegistry.cs:149 trace: [GetProviderAsync] Performing auto-detection of host provider. 09:49:50.317282 ...viderRegistry.cs:158 trace: [GetProviderAsync] Auto-detect probe timeout is 2 ms. 09:49:50.319400 ...viderRegistry.cs:166 trace: [GetProviderAsync] Checking against 3 host providers registered with priority 'Normal'. 09:49:50.320539 ...GitCommandBase.cs:50 trace: [ExecuteAsync] Host provider 'Bitbucket' was selected. 09:49:50.325640 ...tHostProvider.cs:307 trace: [GetSupportedAuthenticationModes] https://bitbucket.org/ is bitbucket.org - authentication schemes: 'All' 09:49:50.330490 ...tHostProvider.cs:103 trace: [GetStoredCredentials] Look for existing credentials under https://bitbucket.org ... 09:49:50.347891 ...tHostProvider.cs:113 trace: [GetStoredCredentials] Found stored credentials: hickford/******** 09:49:50.349173 ...tHostProvider.cs:410 trace: [ValidateCredentialsWork] Validate credentials (hickford/********) are fresh for https://bitbucket.org/ ... 09:49:50.356598 ...tbucketRestApi.cs:74 trace: [GetUserInformationAsync] HTTP: GET https://api.bitbucket.org/2.0/user 09:49:50.358576 ...pClientFactory.cs:58 trace: [CreateClient] Creating new HTTP client instance... 09:49:51.086436 ...tbucketRestApi.cs:77 trace: [GetUserInformationAsync] HTTP: Response 401 [Unauthorized] 09:49:51.088938 ...tHostProvider.cs:433 trace: [ValidateCredentialsWork] Failed to validate existing credentials using OAuth 09:49:51.091501 ...tbucketRestApi.cs:74 trace: [GetUserInformationAsync] HTTP: GET https://api.bitbucket.org/2.0/user 09:49:51.326517 ...tbucketRestApi.cs:77 trace: [GetUserInformationAsync] HTTP: Response 401 [Unauthorized] 09:49:51.327476 ...tHostProvider.cs:447 trace: [ValidateCredentialsWork] Failed to validate existing credentials using Basic Auth 09:49:51.335883 ...tHostProvider.cs:126 trace: [GetRefreshedCredentials] Refresh credentials... 09:49:51.343840 ...tHostProvider.cs:131 trace: [GetRefreshedCredentials] Checking for refresh token... 09:49:51.351093 ...tHostProvider.cs:207 trace: [GetRefreshedCredentials] Found stored refresh token: ******** 09:49:51.356465 ...tHostProvider.cs:229 trace: [GetOAuthCredentialsViaRefreshFlow] Refreshing OAuth credentials using refresh token... 09:49:51.361625 ...pClientFactory.cs:58 trace: [CreateClient] Creating new HTTP client instance... 09:49:52.093797 ...tHostProvider.cs:234 trace: [GetOAuthCredentialsViaRefreshFlow] Resolving username for refreshed OAuth credential... 09:49:52.093931 ...tbucketRestApi.cs:74 trace: [GetUserInformationAsync] HTTP: GET https://api.bitbucket.org/2.0/user 09:49:52.275182 ...tbucketRestApi.cs:77 trace: [GetUserInformationAsync] HTTP: Response 200 [OK] 09:49:52.317717 ...tHostProvider.cs:236 trace: [GetOAuthCredentialsViaRefreshFlow] Username for refreshed OAuth credential is 'hickford' 09:49:52.317785 ...tHostProvider.cs:239 trace: [GetOAuthCredentialsViaRefreshFlow] Storing new refresh token... 09:49:52.322585 ...GitCommandBase.cs:54 trace: [ExecuteAsync] End 'get' command... remote: Forbidden fatal: unable to access 'https://bitbucket.org/chromiumembedded/cef-project.git/': The requested URL returned error: 403
Interesting. Do you know if this issue
- occurs without
credential.credentialstore=cache - occurs with the GUI flow?
Your fix code implies it happens because the flow finds the stored value under /refresh-token and treats it as an access_token. But your code only changes the name the refresh_token is stored under. Why does that fix how the refresh_token is used?
I think I'm missing something. Cheers
Pardon me, I haven't explained well. The problem is that git-credential-manager queries for working credential (missing) but retrieves refresh token by mistake.
In practice this problem only happens with GCM and credential.credentialstore=cache
git clone https://bitbucket.org/chromiumembedded/cef-project.git
cd cef-project
git push
# choose OAuth, push will fail, only refresh token will be stored
# query for credential (expected to fail) returns refresh token by mistake
echo url=https://bitbucket.org/\n | git credential-cache get
# query for refresh token
echo url=https://bitbucket.org/refresh_token\n | git credential-cache get
Ideal solution from perspective of gitcredential would be to store single credential with refresh_token as an extra key https://git-scm.com/docs/git-credential#IOFMT
The credential is split into a set of named attributes, with one attribute per line. Each attribute is specified by a key-value pair.
However this isn't compatible with GCM's ICredentialStore which has a different contract and only supports storing username and password, not list of arbitrary keys. This makes sense because most other stores can only store a username and password and not arbitrary key value pairs.
Ideal solution from perspective of gitcredential would be to store single credential with refresh_token as an extra key https://git-scm.com/docs/git-credential#IOFMT
The credential is split into a set of named attributes, with one attribute per line. Each attribute is specified by a key-value pair.
However this isn't compatible with GCM's ICredentialStore which has a different contract and only supports storing username and password, not list of arbitrary keys. This makes sense because most other stores can only store a username and password and not arbitrary key value pairs.
I think this may be a better solution in the long run, that is to rethink the ICredentialStore and ICredential interface to support different types of credentials and store this along with other associated metadata (like expiration, scopes, etc).
I had planned on sitting down to think about this design for a while now, but I've not yet found the time 😉 . Did you have any ideas or thoughts about this? The main problem is we need to support all the different backing stores relatively equally to store this metadata.
**********@.THIS.IS.CAUSING.THE.ISSUE.FOR.THE.ATM.IN.TYLER.LOCATION...CARD.READER.(RECENT,& HIGHLY ILLEGAL I MIGHT ADD INSTALLED ON THE INSIDE OF THE ATM MACHINE)[email protected],NOT.PRIVATE.PLUS,FOR.LACK.OF.A.BETTER.WORDS,WHO.SOFT.PULLED.THIS.TRAIN.WRECK.OF.A.PACKAGE.IN.THE.FIRST.PLACE TODAY??????????????????????????
Did you have any ideas or thoughts about this? The main problem is we need to support all the different backing stores relatively equally to store this metadata.
Upstream Git patch https://lore.kernel.org/git/[email protected]/ (needs review) adds an oauth_refresh_token attribute to the credential protocol. If merged, it remains to plumb the attribute through GCM and add support to backing stores.
