Node with empty disco_key in DB prevents to start headscale

[winterheart]

Bug description

On updating headscale from 0.22.3 to 0.23.0-alpha5 headscale fails to starts due incorrect entry in database. One of nodes had empty disco_key ("disco_key:"). This entry is causing to headscale fail to start:

headscale Cannot get nodes: failed to unmarshal disco key from db: key hex has the wrong size, got 0 want 64

After manually deleting that node in database headscale starts normally. I have no idea how I got that node in database, but in 0.22.3 this error is not critical in order to start headscale.

Environment

• OS: Linux (Docker container)
• Headscale version: 0.23.0-alpha5
• Tailscale version: 1.58

• Headscale is behind a (reverse) proxy
• Headscale runs in a container

To Reproduce

Have node with empty disco_key in database, restart Headscale.

Logs and attachments

[gabe565]

@kradalby I know this issue has been resolved for a while, but I have two separate Headscale instances that I upgraded to v0.23.0 today. One upgrade went smoothly, but the other logged the error repeatedly for all clients:

2024-09-18T17:54:17Z INF home/runner/work/headscale/headscale/hscontrol/poll.go:705 > node has connected, mapSession: 0xc00015e600, chan: 0xc000444380 node=... node.id=26 omitPeers=false readOnly=false stream=true
2024-09-18T17:54:17Z ERR home/runner/work/headscale/headscale/hscontrol/poll.go:725 > Could not get the create map update error="unmarshalling disco key from db: key hex has the wrong size, got 0 want 64" node=... node.id=26 omitPeers=false readOnly=false stream=true
2024-09-18T17:54:17Z INF home/runner/work/headscale/headscale/hscontrol/poll.go:705 > node has disconnected, mapSession: 0xc00015e600, chan: 0xc000444380 node=... node.id=26 omitPeers=false readOnly=false stream=true

I really appreciate the work you do! I'm really excited for v0.23.0.

Edit: Interestingly, the same error was logged during startup.

2024-09-18T17:54:13Z INF Opening database database=postgres path="host=postgresql-rw dbname=headscale user=headscale"
2024-09-18T17:54:13Z ERR Error accessing db error="unmarshalling disco key from db: key hex has the wrong size, got 0 want 64"
2024-09-18T17:54:14Z INF Setting up a DERPMap update worker frequency=86400000
2024-09-18T17:54:14Z WRN Listening without TLS but ServerURL does not start with http://
2024-09-18T17:54:14Z INF listening and serving HTTP on: 0.0.0.0:8080
2024-09-18T17:54:14Z INF listening and serving debug and metrics on: 0.0.0.0:9090

I wonder if the migration from machines to nodes failed because of the error?

Another Edit: Yep, I think the migration may have failed. I noticed there was a device that hadn't connected in ~5 months, and it had an empty disco_key. I removed that device before the upgrade, and everything is working this time around!