[EKS] [request]: Support affinity in kube-proxy custom configuration

[caleb-devops]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
Please add support for setting affinity to the kube-proxy addon configuration values.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
I use Terraform to bootstrap EKS clusters, and I'd like to be able to disable kube-proxy when the cluster is provisioned to use alternatives like CIlium. The Terraform Kubernetes provider does not support patching existing Kubernetes manifests, so kube-proxy must be manually disabled.

By supporting affinity in the kube-proxy configuration values, I could effectively disable kube-proxy by setting a required affinity key that does not exist on the node.

Example

resource "aws_eks_addon" "kube_proxy" {
  cluster_name = "mycluster"
  addon_name   = "kube-proxy"

  configuration_values = jsonencode({
    affinity = {
      nodeAffinity = {
        requiredDuringSchedulingIgnoredDuringExecution = {
          nodeSelectorTerms = [{
            matchExpressions = [{
              key      = "io.cilium/aws-node-enabled"
              operator = "In"
              values   = ["true"]
            }]
          }]
        }
      }
    }
  })
}

Are you currently working around this issue?
Disable kube-proxy after the cluster has been provisioned.

[mikestef9]

We are solving this (soon) instead through #923