cloudfoundry
cf login with certain options results in rename Access is denied error message
Please fill out the issue checklist below and provide ALL the requested information.
- [x] I reviewed open and closed github issues that may be related to my problem.
- [x] I tried updating to the latest version of the CF CLI to see if it fixed my problem.
- [x] I attempted to run the command with
CF_TRACE=1to help debug the issue. - [x] I am reporting a bug that others will be able to reproduce. ~~- [ ] If this is an issue for the v7 beta release, I've read through the official docs and the release notes.~~
Describe the bug and the command you saw an issue with
Trying to run cf login -a <api url> -o <org> -s Dev --sso but it does not work and instead returns Error writing config: rename C:\Users\<username>\.cf\temp-config696932450 C:\Users\<username>\.cf\config.json: Access is denied.
What happened
C:\Users\<username>\.cf\temp-config696932450 and C:\Users\<username>\.cf\config.json are left on file system. Inspecting config.json it looks incomplete. Manually moving temp-config696932450 to config.json appears to result in it working until next login.
Expected behavior This command logs in without errors
To Reproduce
- Run
cf login -a <api url> -o <org> -s Dev --sso - Enter Temporary Authentication Code
- See error
Provide more context
- Windows 10
cf loginworked in this way through 6.46.1. Starting with 6.47.1 this no longer works.- note 6.47 refactored how
cf loginworks which probably broke this - see https://github.com/cloudfoundry/cli/releases/tag/v6.47.0 - Git bash, PowerShell, CMD... all same error
- still broken in 6.51.0
- API version: 3.66.0
- I have confirmed with multiple users they have same issue
- I have tried to catch what might be locking
config.jsonusing Process Explorer, Resource Monitor, etc. but none seem to catch anything lockingconfig.json
We have created an issue in Pivotal Tracker to manage this:
https://www.pivotaltracker.com/story/show/172734111
The labels on this github issue will be updated when the story is started.
cc @bwasmith @ericpromislow @vitreuz @XenoPhex looks like you all touched https://github.com/cloudfoundry/cli/blob/master/util/configv3/write_config.go where I think this error is getting hit.
Hey there @bbodenmiller,
Thanks for reaching out, someone on the CLI team will reach out at some point to start debugging this issue. But for future reference, all the people you @'d in that message no longer work on that project; but when you create a new issue, the cf-gitbot you see above alerts the right people.
Thank you for reporting the issue, the CLI team will schedule and look into the issue.
Hello @bbodenmiller
Sorry for the delay in getting back to you on this, unfortunately we tried a bunch and could not reproduce this on my Windows 10 machine, running the cf CLI v6.51.0
$ cf -v
cf.exe version 6.51.0+2acd15650.2020-04-07
$ cf login -s s -o o -a https://api.arsicault.cli.fun --skip-ssl-validation --sso
API endpoint: https://api.arsicault.cli.fun
Temporary Authentication Code ( Get one at https://login.arsicault.cli.fun/passcode ): PkiI1PkHRp
Authenticating...
OK
Targeted org o
Targeted space s
API endpoint: https://api.arsicault.cli.fun (API version: 3.87.0)
User: nwebb
Org: o
Space: s
Does this always happen for you? Or is it intermittent, have you seen this issue on other Windows machines, or just yours?
Thanks! @JenGoldstrich and @nickjameswebb
@JenGoldstrich @nickjameswebb yes I am still getting this issue almost always on multiple machines as are other folks at my organization.
$ cf version
cf.exe version 6.52.0+b086fe522.2020-08-26
$ cf login -a <ur> -o <org> -s Dev --sso
API endpoint: <ur>
Temporary Authentication Code ( Get one at <ur>/passcode ): xxxxxxx
Authenticating...
OK
Targeted org <org>
Targeted space Dev
API endpoint: <ur> (API version: 2.139.0)
User: <me>
Org: <org>
Space: Dev
Error writing config: rename C:\Users\<user>\.cf\temp-config156273826 C:\Users\<user>\.cf\config.json: Access is denied.
It sometimes works if I do this but generally still fails with same error most the time
$ cf api <url>
$ cf login -o <org> -s Dev --sso
This seems to work every time but requires interaction:
$ cf login -a <url> --sso
Hi @bbodenmiller, I've done a lot more digging on this issue and discovered that this problem is fairly common on Golang windows projects writing to files (see https://github.com/golang/go/issues/36568, https://github.com/microsoft/vscode-go/issues/2897, https://github.com/golang/go/issues/29424 for similar issues). I think you had the right idea watching for locks on the config, but according to this comment on a Golang issue this error occurs when any process has opened the file, not just when a lock is created.
With that in mind, I've been able to reproduce the error by running multiple cf logins in parallel, but not with a single call to cf login. I'm going to dig deeper into our code to try to find places where we may be opening the file twice in parallel, but in the meantime, I wonder if it's possible that you have an antivirus on your machine that is opening/scanning our temp files, or if you are running other cf commands in the background.
@nickjameswebb I'm not running another other commands but my company does force us to run antivirus.
Hi @bbodenmiller,
We’ve discovered a few problems with the way we handle writing to config.json that seem related to the error you are seeing. However, we still haven’t been able to reproduce the error using your cf login workflow, we aren’t sure that the bugs we've found are the actual underlying cause of your issue, and we obviously can't test against your antivirus. Would you be comfortable with running a Dev build of the CLI on your machine to test whether our changes fix your issue? We can provide a binary or you can build from a branch.
@nickjameswebb yep. A dev build would be best for me.
On Fri, Oct 9, 2020, 10:44 AM Nick Webb [email protected] wrote:
Hi @bbodenmiller https://github.com/bbodenmiller,
We’ve discovered a few problems with the way we handle writing to config.json that seem related to the error you are seeing. However, we still haven’t been able to reproduce the error using your cf login workflow, we aren’t sure that the bugs we've found are the actual underlying cause of your issue, and we obviously can't test against your antivirus. Would you be comfortable with running a Dev build of the CLI on your machine to test whether our changes fix your issue? We can provide a binary or you can build from a branch.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cloudfoundry/cli/issues/1933#issuecomment-706316802, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJDGTGVXHNSBJK3NTJNGZ3SJ5DXNANCNFSM4M363UPQ .
Hi @bbodenmiller here is a dev build with locking implemented on config.json. It also removes the need for a temp-config. Let us know what you see! If it fails please include a full CF_TRACE=true cf login ... log for us to debug.
dev-build.zip
@nickjameswebb as best as I can tell this fixes the issue. I haven't been able to reproduce Error writing config: rename C:\Users\<username>\.cf\temp-config696932450 C:\Users\<username>\.cf\config.json: Access is denied. message with dev build when at same time I was sometimes able to with installed version 6.52.0+b086fe522.2020-08-26.
@bbodenmiller that's good to know, thanks for running that. We are going to produce one more dev build, this time keeping the temp-config code in order to use atomic file writing on Linux. Let me know if you are willing to run another build!
Sounds good.
On Wed, Oct 14, 2020, 11:05 AM Nick Webb [email protected] wrote:
@bbodenmiller https://github.com/bbodenmiller that's good to know, thanks for running that. We are going to produce one more dev build, this time keeping the temp-config code in order to use atomic file writing on Linux. Let me know if you are willing to run another build!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cloudfoundry/cli/issues/1933#issuecomment-708569602, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJDGTE5UDZGZ5FO542IDU3SKXR57ANCNFSM4M363UPQ .
Hi @bbodenmiller, We have a few questions (and a potential action) we're hoping you can answer/take for us. And if you'd prefer not to share the information here due to security concerns, we'd be happy to hop on a zoom or phone call with you at your request.
Our questions:
- What anti-virus software is running?
- if possible to disable momentarily in a sandbox/test-env (basically safe) manner - can you disable antivirus and is the issue reproducible under those conditions?
- What file-system is being used and what version?
- Are you able to reproduce this issue in the latest v6 (now 6.53.0) and v7 (v7.1.0) CLI?
@heyjcollins I can't share what anti-virus software I'm using but the contacts at my company that work directly with you folks likely are able to. It also appears there is no way for me personally to temporarily disable it.
Using NTFS... not sure if it has a version # or how to find it?
Able to reproduce issue with cf.exe version 6.53.0+8e2b70a4a.2020-10-01: Error writing config: rename C:\Users\<user>\.cf\temp-config032471562 C:\Users\<user>\.cf\config.json: Access is denied.
I can't test v7. My understanding is we are running latest LTS PCF release which does not support v7.
Warning: Your targeted API's version (3.74.0) is less than the minimum supported API version (3.85.0). Some commands may not function correctly.
Hello @bbodenmiller,
@vitreuz and I prepared a dev build for you to try dev-build-10-16-200-retry.zip this binary is a compiled version of our commit 4b329
We exposed retry delay via the environment variable WRITE_RETRY_TIMEOUT which is defaulted to 50 milliseconds. Our code makes 5 attempts to write the config file. The idea we are exploring is that this retry logic should let 3rd party to release our file.
Please try and let us know how that works.
@a-b looks like it's working
> cf login -o x -s Dev --sso
API endpoint: x
Temporary Authentication Code ( Get one at x ):
Authenticating...
OK
Targeted org x
Targeted space Dev
API endpoint: x (API version: 3.74.0)
User: x
Org: x
Space: Dev
Error writing config: rename C:\Users\me\.cf\temp-config294620618 C:\Users\me\.cf\config.json: Access is denied.
> .\cf-cli_winx64_build_10_16_2020_retry.exe login -o x -s Dev --sso
API endpoint: x
Temporary Authentication Code ( Get one at x ):
Authenticating...
OK
Targeted org x
Targeted space Dev
API endpoint: x (API version: 3.74.0)
User: x
Org: x
Space: Dev
2020/10/21 19:53:16 Retrying to write config file.
@a-b @vitreuz @joshcollins1 @lisaburns any idea when this fix may land in a 6.x branch & release? It's been nearly 9 months since the fix was built.
Hey @bbodenmiller
Sorry for your delay in getting a response.
Please be careful with who you're pinging as most of the folks you've pinged no longer work on CloudFoundry.
Currently with v6 my impression is that the team is only releasing critical bug fixes, we have planned releases going forward for the next few months relating to v7. We will try to prioritize a v6 release in the future, but there are no immediate plans to release v6.
Thanks, Jenna