IP address for VM is not properly determined

[0xff8]

Hi, guys,

After the upgrade to the latest version 2.23.0 of the tart and tart-gitlab-executor 1.21.0-7973405 and pulling latest ubuntu image I am having a problem
The command tart ip shows wrong IP address for a virtual machine and pipelines stopped to work.

Please see the screenshot:

Any clue what might be wrong?

[edigaryev]

Unfortunately macOS's DHCP server is treating hw_address and identifier fields in /var/db/dhcpd_leases the same way and putting the client identifier, which is a DUID/IAID in your case, into the hw_address field, making it impossible for tart ip to locate the corresponding entry given a MAC-address.

You need to disable DUID/IAID/non-MAC identifiers in your VM to work around that.

See cirruslabs/linux-image-templates#39 for more details.

[0xff8]

Thanks @edigaryev for the response. Could it be disabled in the base image then ghcr.io/cirruslabs/ubuntu:latest ? Previous version of that image was working fine..

[0xff8]

Test with the macOS image - same problem:

2025/01/15 16:18:24 Pulling the latest version of ghcr.io/cirruslabs/macos-ventura-xcode:14...
2025/01/15 16:18:25 Cloning and configuring a new VM...
2025/01/15 16:18:25 Waiting for the VM to boot and be SSH-able...

[edigaryev]

Previous version of that image was working fine

Hmm, indeed. It seems that 99_cirruslabs.cfg didn't make it to the VM for some reason.

Could you please re-pull the image? It should work now.

[edigaryev]

Test with the macOS image - same problem:

This is unlikely related. Are you running macOS 15 (Sequoia) on the host by a chance?

If so, please see the top of the Gitlab Tart Executor's README.md explaining this issue.

[0xff8]

Yes, I am running macOS 15.2 . I have installed GitLab Runner latest version I have changed the security options for it according the readme.md:

but still machine stuck :/

Using Custom executor...
2025/01/15 18:03:56 Pulling the latest version of ghcr.io/cirruslabs/macos-ventura-xcode:14...
2025/01/15 18:03:56 Cloning and configuring a new VM...
2025/01/15 18:03:56 Waiting for the VM to boot and be SSH-able...

[amsokol]

Hello,

If it is still relevant, to make tart ip <vm> work for ubuntu, it necessary to add dhcp-identifier: "mac" to the /etc/netplan/50-cloud-init.yaml and restart. Example:

network:
  version: 2
  ethernets:
    enp0s1:
      dhcp4: true
      dhcp-identifier: "mac"

tested with ubuntu 24.10

[edigaryev]

but still machine stuck :/

1. Have you applied the DHCP fix?
2. Try adding TART_EXECUTOR_HEADLESS set to false to your .gitlab-ci.yml. Connect to the host using Screen Sharing, start the job and check whether the VM boots at all and if so, which IP address does it get (System Settings → Network → Ethernet from inside of a VM).

[gitperr]

Something similar is happening to us.

Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start." UserInfo={NSLocalizedFailure=Internal Virtualization error., NSLocalizedFailureReason=The virtual machine failed to start.}
Failed to retrieve IP address of VM "gitlab-574421" in 60 seconds:
tart command returned non-zero exit code: "no IP address found, is your VM running?", will re-try...Failed to retrieve IP address of VM "gitlab-574421" in 60 seconds: tart command returned non-zero exit code: "no IP address found, is your VM running?"

tart --version
2.23.0
gitlab-tart-executor --version          
executor version 1.21.0-7973405
gitlab-runner --version
Version:      17.7.0
Git revision: 3153ccc6
Git branch:   17-7-stable
GO version:   go1.23.4
Built:        2024-12-19T19:55:28+00:00
OS/Arch:      darwin/arm64

Additionally, all of these have the LC_UUID.

Both the VM image I intend to run, and the host are on macOS 15.2 (Sequoia):

sw_vers
ProductName:		macOS
ProductVersion:	15.2
BuildVersion:		24C101

When the pipeline is started and tart executor prints:

Waiting for the VM to boot and be SSH-able...

I go into the runner host and check the list of VMs: tart list, but no VMs are running. Additionally, I've never seen a VM called gitlab-574421 anywhere.

• I do see my own built VM image though (Sequoia image mentioned earlier), I can run it as normal, and SSH to it too.
• We are able to run the pipelines as usual with macOS 14.x (Sonoma)

[edigaryev]

@gitperr this looks like unrelated issue to me. Here the VM is clearly not running (because it has failed), so it won't be possible to retrieve its IP address.

Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start." UserInfo={NSLocalizedFailure=Internal Virtualization error., NSLocalizedFailureReason=The virtual machine failed to start.}

Are you running GitLab Tart Executor/Tart as root by a chance?

Or starting it via launchd without <key>UserName</key> being set?

[gitperr]

Gitlab runner is running as root, so I imagine forked processes also start under root.

I checked the .plist file: /Library/LaunchDaemons/gitlab-runner.plist because it is starting under the launchd, and it does not have the key you mentioned. I had installed it with sudo gitlab-runner --install --user <user> and it seems to not add that key.

PS: I retried without root and it seems to work now.

[0xff8]

but still machine stuck :/

1. Have you applied the DHCP fix?
2. Try adding TART_EXECUTOR_HEADLESS set to false to your .gitlab-ci.yml. Connect to the host using Screen Sharing, start the job and check whether the VM boots at all and if so, which IP address does it get (System Settings → Network → Ethernet from inside of a VM).

Thanks @edigaryev !

1. it was never a problem, since we run the machine quite rarely , but I have applied it now
2. Thanks for this advice . I will attempt to do so.

@gitperr I do not run tart is a root neither gitlab-runner.

[0xff8]

I can confirm that I have everything working now.
The only change I have made - applied DHCP fix .
Do you consider adding the fix to homebrew installation script?

[edigaryev]

I can confirm that I have everything working now. The only change I have made - applied DHCP fix . Do you consider adding the fix to homebrew installation script?

Applying it requires superuser privileges, which Homebrew doesn't seem to support.

But we can probably add a caveats section 🤔

[0xff8]

Yeah, you could add a sudo command in post-install , or at least mention this importation information in after-installation hint.
thanks again, everything is working. Shall I close the issue?

[edigaryev]

Yeah, you could add a sudo command in post-install

That's not always possible because by default sudo is configured to ask for a password, which is not something we want to have for automated installs.