Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node stale in NotReady after copying data to new disk #11654

Closed
LMSSonos opened this issue Jan 25, 2025 · 0 comments
Closed

Node stale in NotReady after copying data to new disk #11654

LMSSonos opened this issue Jan 25, 2025 · 0 comments

Comments

@LMSSonos
Copy link

Environmental Info:
K3s Version:
k3s version v1.31.4+k3s1 (a562d09)
go version go1.22.

Node(s) CPU architecture, OS, and Version:
Linux pi5-node3 6.6.62+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.62-1+rpt1 (2024-11-25) aarch64 GNU/Linux

Cluster Configuration:
3 nodes

NAME        STATUS     ROLES                       AGE   VERSION
node1   Ready      control-plane,etcd,master   31h   v1.31.4+k3s1
node2   Ready      control-plane,etcd,master   31h   v1.31.4+k3s1
node3   NotReady   control-plane,etcd,master   31h   v1.31.4+k3s1

Describe the bug:
In node 3 I used dd to copy the data from the sdcard to a ssd (raspberry). After the reboot, the k3s will not start anymore.

k3s describe ndoe returns this issue

Conditions:

  Type             Status    LastHeartbeatTime                 LastTransitionTime                Reason              Message
  ----             ------    -----------------                 ------------------                ------              -------
  EtcdIsVoter      True      Sat, 25 Jan 2025 17:23:44 +0100   Fri, 24 Jan 2025 09:58:35 +0100   MemberNotLearner    Node is a voting member of the etcd cluster
  MemoryPressure   Unknown   Sat, 25 Jan 2025 17:01:59 +0100   Sat, 25 Jan 2025 17:07:29 +0100   NodeStatusUnknown   Kubelet stopped posting node status.
  DiskPressure     Unknown   Sat, 25 Jan 2025 17:01:59 +0100   Sat, 25 Jan 2025 17:07:29 +0100   NodeStatusUnknown   Kubelet stopped posting node status.
  PIDPressure      Unknown   Sat, 25 Jan 2025 17:01:59 +0100   Sat, 25 Jan 2025 17:07:29 +0100   NodeStatusUnknown   Kubelet stopped posting node status.
  Ready            Unknown   Sat, 25 Jan 2025 17:01:59 +0100   Sat, 25 Jan 2025 17:07:29 +0100   NodeStatusUnknown   Kubelet stopped posting node status.

journalctl -xeu k3s.service

Jan 25 17:37:58 node3 k3s[7788]: created by go.etcd.io/etcd/server/v3/etcdserver.openBackend in goroutine 1
Jan 25 17:37:58 node3 k3s[7788]:         /go/pkg/mod/github.com/k3s-io/etcd/server/[email protected]/etcdserver/backend.go:75 +0x120
Jan 25 17:37:58 node3 systemd[1]: k3s.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ An ExecStart= process belonging to unit k3s.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 2.
Jan 25 17:37:58 node3 systemd[1]: k3s.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Jan 25 17:37:58 node3 systemd[1]: Failed to start k3s.service - Lightweight Kubernetes.
░░ Subject: A start job for unit k3s.service has failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has finished with a failure.
░░ 
░░ The job identifier is 22132 and the job result is failed.

Are there any steps required to make after the drive has changed? Thanks

@k3s-io k3s-io locked and limited conversation to collaborators Jan 25, 2025
@brandond brandond converted this issue into discussion #11655 Jan 25, 2025
@github-project-automation github-project-automation bot moved this from New to Done Issue in K3s Development Jan 25, 2025

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
Status: Done Issue
Development

No branches or pull requests

1 participant