Daemon fail in when launching colima start from launchd user agent

[tanshihaj]

Description

I've faced with strange issue: I cannot launch colima daemon using launchd user agents. It seems that issue somewhere between daemon and some client since I see following error in /Users/tanshihaj/.colima/default/daemon/daemon.log file:

time="2022-11-21T18:30:04+02:00" level=info msg="daemon started by colima"
time="2022-11-21T18:30:04+02:00" level=info msg="Run `pkill -F /Users/tanshihaj/.colima/default/daemon/daemon.pid` to kill the daemon"
time="2022-11-21T18:30:04+02:00" level=info msg="Using search domains: [test.net]"
time="2022-11-21T18:30:04+02:00" level=info msg="waiting for clients..."
time="2022-11-21T18:30:05+02:00" level=info msg="new connection from  to /Users/tanshihaj/.colima/default/daemon/gvproxy.sock"
time="2022-11-21T18:30:31+02:00" level=error msg="cannot receive packets from , disconnecting: cannot read size from socket: EOF"
time="2022-11-21T18:30:31+02:00" level=error msg="virtual network err: \"cannot read size from socket: EOF\""

My user agent /Users/tanshihaj/Library/LaunchAgents/org.colima.daemon-starter.plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>EnvironmentVariables</key>
	<dict>
		<key>PATH</key>
		<string>/nix/store/3na28jnhikj3siq1vyjq84ggl711s82x-colima-0.4.6/bin:/nix/store/fg3xj4i9vvica9yzs0zqx1y0kgnnba24-docker-20.10.21/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
	</dict>
	<key>KeepAlive</key>
	<false/>
	<key>Label</key>
	<string>org.colima.daemon-starter</string>
	<key>ProgramArguments</key>
	<array>
		<string>/nix/store/3na28jnhikj3siq1vyjq84ggl711s82x-colima-0.4.6/bin/colima</string>
		<string>start</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	<key>StandardErrorPath</key>
	<string>/Users/tanshihaj/stderr.log</string>
	<key>StandardOutPath</key>
	<string>/Users/tanshihaj/stdout.log</string>
</dict>
</plist>

Does anyone have an idea what goes wrong?

Version

Colima Version: 0.4.6/10377f3
Lima Version: 0.13.0
Qemu Version: 7.1.0

Operating System

• macOS Intel
• macOS M1
• Linux

Reproduction Steps

1. Create user agent file/Users/username/Library/LaunchAgents/org.colima.daemon-starter.plist with content above (substitute correct path to your home, colima path and docker client path)
2. Load agent: launchctl load /Users/username/Library/LaunchAgents/org.colima.daemon-starter.plist
3. Try to start it: launchctl kickstart -k gui/$(id -u)/org.colima.daemon-starter
4. Check that colima didn't started: colima status

Expected behaviour

I'm expecting that colima starts on background

Additional context

It seems if I run colima start in user shell daemon runs normally.

[AlexeyRaga]

I am having the same issue and frustration on MacOS Intel.
@tanshihaj were you able to solve it or find a workaround?

[tanshihaj]

Unfortunately not(

[lloeki]

Looks like a sandboxing issue.

When colima is started for the terminal it inherits Terminal.app sandbox settings, which are much wider (notably if the Developer Mode permission is checked); mosh is having possibly similar issues at mobile-shell/mosh#1254 (comment)

Otherwise, it should trigger a UI prompt for the user to allow, say, network access for that† binary. Unfortunately it seems as if starting from a LaunchAgent does not trigger showing the UI prompt.

† If the binary changes (path or content, say on an update) the permission is invalidated.

[lloeki]

Also given this attempt is using Nix and there's a Homebrew LaunchDaemon that appears to have worked at some point, it may be worth comparing how both behave today.

In any case, I opened an issue at nix-darwin: LnL7/nix-darwin#1182

#262 would still be useful for users that are not using nix-darwin, and this very issue right here would still need to be addressed/explained in some way.

[lloeki]

Well, this ~/Library/LaunchAgents/org.colima.default.plist worked quite well for me:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>EnvironmentVariables</key>
	<dict>
		<key>PATH</key>
		<string>/nix/store/3na28jnhikj3siq1vyjq84ggl711s82x-colima-0.4.6/bin:/nix/store/s80wnl01gsw7a6bmm8p0r5daizb1k629-docker-25.0.6/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
	</dict>
	<key>KeepAlive</key>
	<true/>
	<key>Label</key>
	<string>org.colima.default</string>
	<key>ProgramArguments</key>
	<array>
		<string>/bin/sh</string>
		<string>-c</string>
		<string>/bin/wait4path /nix/store &amp;&amp; exec /nix/store/8w9b9jym9ab86nz9yy2lv1795yxq6ylj-colima-0.6.9/bin/colima start --foreground</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	<key>StandardErrorPath</key>
    <string>/Users/loic.nageleisen/.colima/default/daemon/launchd.stderr.log</string>
	<key>StandardOutPath</key>
    <string>/Users/loic.nageleisen/.colima/default/daemon/launchd.stdout.log</string>
	<key>WorkingDirectory</key>
    <string>/Users/loic.nageleisen</string>
</dict>
</plist>

launchctl load -w ~/Library/LaunchAgents/org.colima.default.plist

~/.colima/default/daemon/launchd.stderr.log:

time="2024-11-19T11:15:26+01:00" level=info msg="provisioning ..." context=docker
time="2024-11-19T11:15:26+01:00" level=info msg="starting ..." context=docker
time="2024-11-19T11:15:27+01:00" level=info msg=done
time="2024-11-19T11:15:27+01:00" level=info msg="keeping Colima in the foreground, press ctrl+c to exit..."

~/.colima/default/daemon/launchd.stdout.log:

time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Waiting for the essential requirement 1 of 2: \"ssh\""
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] The essential requirement 1 of 2 is satisfied"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Waiting for the essential requirement 2 of 2: \"user session is ready for ssh\""
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] The essential requirement 2 of 2 is satisfied"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Waiting for the guest agent to be running"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Forwarding \"/var/run/docker.sock\" (guest) to \"/Users/loic.nageleisen/.colima/default/docker.sock\" (host)"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Forwarding \"/var/run/docker.sock\" (guest) to \"/Users/loic.nageleisen/.colima/docker.sock\" (host)"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Guest agent is running"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Waiting for the final requirement 1 of 1: \"boot scripts must have finished\""
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Not forwarding TCP [::]:22"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Not forwarding TCP 127.0.0.54:53"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] Not forwarding TCP 127.0.0.53:53"
time="2024-11-19T11:15:25+01:00" level=info msg="[hostagent] The final requirement 1 of 1 is satisfied"
time="2024-11-19T11:15:25+01:00" level=info msg="READY. Run `limactl shell colima` to open the shell."
  File: /proc/sys/fs/binfmt_misc/rosetta
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 0,44	Inode: 6599        Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-11-19 11:15:21.809000002 +0100
Modify: 2024-11-19 11:15:21.809000002 +0100
Change: 2024-11-19 11:15:21.809000002 +0100
 Birth: -
colima
Successfully created context "colima"
colima
Current context is now "colima"

~/.colima/default/daemon/daemon.log:

time="2024-11-19T11:15:14+01:00" level=info msg="- - - - - - - - - - - - - - -"
time="2024-11-19T11:15:14+01:00" level=info msg="daemon started by colima"
time="2024-11-19T11:15:14+01:00" level=info msg="Run `/usr/bin/pkill -F /Users/loic.nageleisen/.colima/default/daemon/daemon.pid` to kill the daemon"
time="2024-11-19T11:15:14+01:00" level=info msg="waiting for VM to start" context=inotify
time="2024-11-19T11:15:14+01:00" level=info msg="waiting 5 secs for VM" context=inotify
time="2024-11-19T11:15:19+01:00" level=info msg="waiting 5 secs for VM" context=inotify
time="2024-11-19T11:15:25+01:00" level=info msg="VM started" context=inotify

[lloeki]

And with nix-darwin:

  launchd.agents."colima.default" = {
    command = "${pkgs.colima}/bin/colima start --foreground";
    serviceConfig = {
      Label = "com.colima.default";
      RunAtLoad = true;
      KeepAlive = true;

      # not sure where to put these paths and not reference a hard-coded `$HOME`; `/var/log`?
      StandardOutPath = "/Users/loic.nageleisen/.colima/default/daemon/launchd.stdout.log";
      StandardErrorPath = "/Users/loic.nageleisen/.colima/default/daemon/launchd.stderr.log";

      # not using launchd.agents.<name>.path because colima needs the system ones as well
      EnvironmentVariables = {
        PATH = "${pkgs.colima}/bin:${pkgs.docker}/bin:/usr/bin:/bin:/usr/sbin:/sbin";
      };
    };
  };

$ darwin-rebuild switch
$ launchctl load -w /Library/LaunchAgents/com.colima.default.plist
$ launchctl print gui/$(id -u)/com.colima.default
$ colima status

Note that you may have some sandbox allowance popups the first time you access some directories from within the VM.