Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: writing tilt api configs: open /home/$USER/.tilt-dev/config.lock: file exists #4814

Closed
landism opened this issue Jul 29, 2021 · 12 comments · Fixed by #5982
Closed

Error: writing tilt api configs: open /home/$USER/.tilt-dev/config.lock: file exists #4814

landism opened this issue Jul 29, 2021 · 12 comments · Fixed by #5982
Labels
bug Something isn't working

Comments

@landism
Copy link
Member

landism commented Jul 29, 2021

Two users have reported getting this error when starting Tilt:
Error: writing tilt api configs: open /home/$USER/.tilt-dev/config.lock: file exists

We don't have a repro.

The issue can be worked around by deleting the config.lock file.

@landism landism added the bug Something isn't working label Jul 29, 2021
@landism
Copy link
Member Author

landism commented Jul 30, 2021

FWICT, this locking only happens here.

I haven't managed to repro by running tilt a ton, and I haven't come up with any good theories as to the cause by inspecting the code.

My current best guess is:

  1. When the user ctrl+C's Tilt, it tells all its subscribers to teardown, which includes removing the tilt api config from the config file.
  2. If the user hits ctrl+C a second time, or if teeardown takes 2 seconds, Tilt will stop waiting and just exit immediately.
  3. We have a subscriber that edits the api config on teardown
  4. This could potentially cause this issue if (2) happens while the config writer holds the lock. However, when testing, it seemed like it always gave up the lock extremely quickly, and I didn't have any luck hitting ctrl+c twice quickly enough to catch it before the teardown completed. My only working theory is that there is some other condition I'm missing that causes the config writer to take more time.

@bahatron
Copy link

bahatron commented Jul 31, 2021

I'm also experiencing tilt taking up to 30seconds to load (from the moment I type tilt up to the chrome tab being able to connect) usually right after having to run sudo rm -rf ~/.tilt-dev/. I'm on Ubuntu 20.04

@nicks
Copy link
Member

nicks commented Jul 31, 2021

@bahatron hmmm! That's interesting, we've never seen that before. Could you file a new bug for that and include the output of 'tilt doctor'? It seems unrelated to this issue, and we may need to collect more info from you to repro

@Kludex
Copy link

Kludex commented Sep 5, 2021

I'm having the same issue, so:

❯ tilt doctor
Tilt: v0.22.6, built 2021-08-27
System: linux-amd64
---
Docker (cluster)
- Host: tcp://192.168.49.2:2376
- Server Version: 20.10.7
- API Version: 1.41
- Builder: 2
---
Docker (local)
- Host: [default]
- Server Version: 20.10.7
- Version: 1.41
- Builder: 2
---
Kubernetes
- Env: minikube
- Context: minikube
- Cluster Name: minikube
- Namespace: default
- Container Runtime: docker
- Version: v1.21.2
- Cluster Local Registry: none
---
Thanks for seeing the Tilt Doctor!
Please send the info above when filing bug reports. 💗

The info below helps us understand how you're using Tilt so we can improve,
but is not required to ask for help.
---
Analytics Settings
--> (These results reflect your personal opt in/out status and may be overridden by an `analytics_settings` call in your Tiltfile)
- User Mode: opt-in
- Machine: f07e6b21bb352bd8a476634d78e325c6
- Repo: zkWI+v7aP9kuPp54+TzNBQ==

After deleting the .tilt-dev directory, it works just fine.

@milas
Copy link
Contributor

milas commented Sep 7, 2021

@Kludex Thanks for the report - do you remember the last time you exited Tilt before the error started? Was it by Ctrl-C in the terminal? Rebooting machine? Has Tilt ever crashed with a panic for you?

@Kludex
Copy link

Kludex commented Sep 7, 2021

I don't recall. Tilt has never crashed with panic for me.

The next person seeing this issue, please check @milas question above! And copy the .tilt-dev to another location, it may be helpful here, I guess. 😅

@andymartin-sch
Copy link
Contributor

we had two people see this today on 0.23.1 (not related to any recent version upgrade) 🤔 will post more details if it happens again

@filipkowal
Copy link

filipkowal commented Dec 3, 2021

I've experienced the same bug. Removing config.lock helped.

Tilt: v0.23.1, built 2021-11-19
System: linux-amd64
---
Docker
- Host: [default]
- Server Version: 20.10.11
- API Version: 1.41
- Builder: 2
---
Kubernetes
- Env: kind-0.6+
- Context: kind-kind
- Cluster Name: kind-kind
- Namespace: default
- Container Runtime: containerd
- Version: v1.21.1
- Cluster Local Registry: {Host:localhost:43059 hostFromCluster: SingleName:}
---

@nicksieger
Copy link
Member

It would be nice if golang/go#33974 happened. We might need to look at a workaround before then.

@kensipe
Copy link
Contributor

kensipe commented Feb 7, 2022

spent a few mins digging into this today... Nicks link to go issue is the best option. The dependency that locks the file for us is client-go. They have a reference as a TODO to use actual file locks: https://github.com/kubernetes/client-go/blob/master/tools/clientcmd/loader.go#L436-L438

func lockFile(filename string) error {
	// TODO: find a way to do this with actual file locks. Will
	// probably need separate solution for windows and Linux.

Analyzing the situation... The best way is to use the OS locks (syscall.Flock). In addition, it would be best if this was an upstream solution in client-go. It seems this tilt issue is more of a tracking issue.

@JamieDanielson
Copy link

Came across this issue for the first time yesterday, after making a few changes to a Tiltfile. Removing the config.lock file fixed the issue. Let me know if any other detail is helpful!

Tilt: v0.23.1, built 2021-11-19
System: darwin-amd64

@douglasg14b
Copy link

I also get this on Ubuntu, after Ctrl+C tilt, and after the system goes to sleep

nicksieger added a commit to nicksieger/tilt that referenced this issue Nov 30, 2022
nicksieger added a commit to nicksieger/tilt that referenced this issue Nov 30, 2022
nicksieger added a commit that referenced this issue Dec 20, 2022
* config: copy go's file lock and use it instead of client-go's

Fixes #4814.

Signed-off-by: Nick Sieger <[email protected]>

* config: attempt to complete windows filelock implementation

Signed-off-by: Nick Sieger <[email protected]>

Signed-off-by: Nick Sieger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.