-
Notifications
You must be signed in to change notification settings - Fork 210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gha: cleanup stale AWS-CNI iptables rules in EKS tunnel workflow #2045
Conversation
Extend the eks-tunnel script to additionally flush the stale AWS-CNI iptables rules left by the aws-node DaemonSet, which break host to pod connectivity. Using the cleanup commands from the documentation [1]. [1]: https://github.com/cilium/cilium/blob/cc2051eec0ba5e57842d8a7d4b4b33f31b12509f/Documentation/installation/k8s-install-helm.rst?plain=1#L208-L211 Signed-off-by: Marco Iorio <[email protected]>
Link to successful run: https://github.com/cilium/cilium-cli/actions/runs/6529848997/job/17728137356?pr=2045 |
c68088e
to
d7a6aa7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought we already did that 🤔
This should be done by this post-start cilium Daemonset hook: https://github.com/cilium/cilium/blob/main/install/kubernetes/cilium/files/agent/poststart-eni.bash Should we update this script to also remove @giorio94 ^^ |
I don't know the details of the poststart-eni script, but makes definitely sense to me (it is very easy to miss the documentation and hit this issue). I edited this workflow because from a quick look at cilium/cilium#25804 it seemed that the script was not always executed. Even if we extend the |
Just double-checked, and it is not run when running on EKS in tunnel mode, because /cc @nebril |
AKS failure is unrelated, marking ready to merge |
This seems like it's fixing CI, but isn't the same problem true in production systems? |
Unfortunately yes. More info in cilium/cilium#25804 |
Ah right. I was gonna comment that this should be fine as long as we document that this is temporary code to work around a bug in Cilium. But I see that the code comment already references that open item. Sure, let's merge this to keep things moving, then we can revisit how to approach that issue for a longer term fix. |
Extend the eks-tunnel script to additionally flush the stale AWS-CNI iptables rules left by the aws-node DaemonSet, which break host to pod connectivity. Using the cleanup commands from the documentation 1.
For #2035.