tilt icon indicating copy to clipboard operation
tilt copied to clipboard

Error: writing tilt api configs: open /home/$USER/.tilt-dev/config.lock: file exists

Open landism opened this issue 3 years ago • 11 comments

Two users have reported getting this error when starting Tilt: Error: writing tilt api configs: open /home/$USER/.tilt-dev/config.lock: file exists

We don't have a repro.

The issue can be worked around by deleting the config.lock file.

landism avatar Jul 29 '21 18:07 landism

FWICT, this locking only happens here.

I haven't managed to repro by running tilt a ton, and I haven't come up with any good theories as to the cause by inspecting the code.

My current best guess is:

  1. When the user ctrl+C's Tilt, it tells all its subscribers to teardown, which includes removing the tilt api config from the config file.
  2. If the user hits ctrl+C a second time, or if teeardown takes 2 seconds, Tilt will stop waiting and just exit immediately.
  3. We have a subscriber that edits the api config on teardown
  4. This could potentially cause this issue if (2) happens while the config writer holds the lock. However, when testing, it seemed like it always gave up the lock extremely quickly, and I didn't have any luck hitting ctrl+c twice quickly enough to catch it before the teardown completed. My only working theory is that there is some other condition I'm missing that causes the config writer to take more time.

landism avatar Jul 30 '21 15:07 landism

I'm also experiencing tilt taking up to 30seconds to load (from the moment I type tilt up to the chrome tab being able to connect) usually right after having to run sudo rm -rf ~/.tilt-dev/. I'm on Ubuntu 20.04

bahatron avatar Jul 31 '21 09:07 bahatron

@bahatron hmmm! That's interesting, we've never seen that before. Could you file a new bug for that and include the output of 'tilt doctor'? It seems unrelated to this issue, and we may need to collect more info from you to repro

nicks avatar Jul 31 '21 20:07 nicks

I'm having the same issue, so:

❯ tilt doctor
Tilt: v0.22.6, built 2021-08-27
System: linux-amd64
---
Docker (cluster)
- Host: tcp://192.168.49.2:2376
- Server Version: 20.10.7
- API Version: 1.41
- Builder: 2
---
Docker (local)
- Host: [default]
- Server Version: 20.10.7
- Version: 1.41
- Builder: 2
---
Kubernetes
- Env: minikube
- Context: minikube
- Cluster Name: minikube
- Namespace: default
- Container Runtime: docker
- Version: v1.21.2
- Cluster Local Registry: none
---
Thanks for seeing the Tilt Doctor!
Please send the info above when filing bug reports. 💗

The info below helps us understand how you're using Tilt so we can improve,
but is not required to ask for help.
---
Analytics Settings
--> (These results reflect your personal opt in/out status and may be overridden by an `analytics_settings` call in your Tiltfile)
- User Mode: opt-in
- Machine: f07e6b21bb352bd8a476634d78e325c6
- Repo: zkWI+v7aP9kuPp54+TzNBQ==

After deleting the .tilt-dev directory, it works just fine.

Kludex avatar Sep 05 '21 07:09 Kludex

@Kludex Thanks for the report - do you remember the last time you exited Tilt before the error started? Was it by Ctrl-C in the terminal? Rebooting machine? Has Tilt ever crashed with a panic for you?

milas avatar Sep 07 '21 14:09 milas

I don't recall. Tilt has never crashed with panic for me.

The next person seeing this issue, please check @milas question above! And copy the .tilt-dev to another location, it may be helpful here, I guess. :sweat_smile:

Kludex avatar Sep 07 '21 15:09 Kludex

we had two people see this today on 0.23.1 (not related to any recent version upgrade) 🤔 will post more details if it happens again

andymartin-sch avatar Dec 02 '21 22:12 andymartin-sch

I've experienced the same bug. Removing config.lock helped.

Tilt: v0.23.1, built 2021-11-19
System: linux-amd64
---
Docker
- Host: [default]
- Server Version: 20.10.11
- API Version: 1.41
- Builder: 2
---
Kubernetes
- Env: kind-0.6+
- Context: kind-kind
- Cluster Name: kind-kind
- Namespace: default
- Container Runtime: containerd
- Version: v1.21.1
- Cluster Local Registry: {Host:localhost:43059 hostFromCluster: SingleName:}
---

filipkowal avatar Dec 03 '21 10:12 filipkowal

It would be nice if golang/go#33974 happened. We might need to look at a workaround before then.

nicksieger avatar Dec 03 '21 16:12 nicksieger

spent a few mins digging into this today... Nicks link to go issue is the best option. The dependency that locks the file for us is client-go. They have a reference as a TODO to use actual file locks: https://github.com/kubernetes/client-go/blob/master/tools/clientcmd/loader.go#L436-L438

func lockFile(filename string) error {
	// TODO: find a way to do this with actual file locks. Will
	// probably need separate solution for windows and Linux.

Analyzing the situation... The best way is to use the OS locks (syscall.Flock). In addition, it would be best if this was an upstream solution in client-go. It seems this tilt issue is more of a tracking issue.

kensipe avatar Feb 07 '22 21:02 kensipe

Came across this issue for the first time yesterday, after making a few changes to a Tiltfile. Removing the config.lock file fixed the issue. Let me know if any other detail is helpful!

Tilt: v0.23.1, built 2021-11-19
System: darwin-amd64

JamieDanielson avatar May 26 '22 17:05 JamieDanielson

I also get this on Ubuntu, after Ctrl+C tilt, and after the system goes to sleep

douglasg14b avatar Oct 15 '22 21:10 douglasg14b