Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add optional list of policy ARNs for attachment to Karpenter IRSA #2537

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 36 additions & 20 deletions examples/karpenter/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,26 @@ module "eks" {
coredns = {
configuration_values = jsonencode({
computeType = "Fargate"
# Ensure that the we fully utilize the minimum amount of resources that are supplied by
# Fargate https://docs.aws.amazon.com/eks/latest/userguide/fargate-pod-configuration.html
# Fargate adds 256 MB to each pod's memory reservation for the required Kubernetes
# components (kubelet, kube-proxy, and containerd). Fargate rounds up to the following
# compute configuration that most closely matches the sum of vCPU and memory requests in
# order to ensure pods always have the resources that they need to run.
resources = {
limits = {
cpu = "0.25"
# We are targetting the smallest Task size of 512Mb, so we subtract 256Mb from the
# request/limit to ensure we can fit within that task
memory = "256M"
}
requests = {
cpu = "0.25"
# We are targetting the smallest Task size of 512Mb, so we subtract 256Mb from the
# request/limit to ensure we can fit within that task
memory = "256M"
}
}
})
}
}
Expand All @@ -109,26 +129,18 @@ module "eks" {
},
]

fargate_profiles = merge(
{ for i in range(3) :
"kube-system-${element(split("-", local.azs[i]), 2)}" => {
selectors = [
{ namespace = "kube-system" }
]
# We want to create a profile per AZ for high availability
subnet_ids = [element(module.vpc.private_subnets, i)]
}
},
{ for i in range(3) :
"karpenter-${element(split("-", local.azs[i]), 2)}" => {
selectors = [
{ namespace = "karpenter" }
]
# We want to create a profile per AZ for high availability
subnet_ids = [element(module.vpc.private_subnets, i)]
}
},
)
fargate_profiles = {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bryantbiggs Is it no longer a recommendation to create one Fargate profile per AZ as noted here?

karpenter = {
selectors = [
{ namespace = "karpenter" }
]
}
kube-system = {
selectors = [
{ namespace = "kube-system" }
]
}
}

tags = merge(local.tags, {
# NOTE - if creating multiple security groups with this module, only tag the
Expand All @@ -148,6 +160,10 @@ module "karpenter" {
cluster_name = module.eks.cluster_name
irsa_oidc_provider_arn = module.eks.oidc_provider_arn

policies = {
AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

tags = local.tags
}

Expand Down
2 changes: 2 additions & 0 deletions modules/karpenter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ No modules.
| [aws_iam_role.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.irsa](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.irsa_additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_sqs_queue.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue) | resource |
| [aws_sqs_queue_policy.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_policy) | resource |
Expand Down Expand Up @@ -169,6 +170,7 @@ No modules.
| <a name="input_irsa_tag_key"></a> [irsa\_tag\_key](#input\_irsa\_tag\_key) | Tag key (`{key = value}`) applied to resources launched by Karpenter through the Karpenter provisioner | `string` | `"karpenter.sh/discovery"` | no |
| <a name="input_irsa_tags"></a> [irsa\_tags](#input\_irsa\_tags) | A map of additional tags to add the the IAM role for service accounts | `map(any)` | `{}` | no |
| <a name="input_irsa_use_name_prefix"></a> [irsa\_use\_name\_prefix](#input\_irsa\_use\_name\_prefix) | Determines whether the IAM role for service accounts name (`irsa_name`) is used as a prefix | `bool` | `true` | no |
| <a name="input_policies"></a> [policies](#input\_policies) | Policies to attach to the IAM role in `{'static_name' = 'policy_arn'}` format | `map(string)` | `{}` | no |
| <a name="input_queue_kms_data_key_reuse_period_seconds"></a> [queue\_kms\_data\_key\_reuse\_period\_seconds](#input\_queue\_kms\_data\_key\_reuse\_period\_seconds) | The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again | `number` | `null` | no |
| <a name="input_queue_kms_master_key_id"></a> [queue\_kms\_master\_key\_id](#input\_queue\_kms\_master\_key\_id) | The ID of an AWS-managed customer master key (CMK) for Amazon SQS or a custom CMK | `string` | `null` | no |
| <a name="input_queue_managed_sse_enabled"></a> [queue\_managed\_sse\_enabled](#input\_queue\_managed\_sse\_enabled) | Boolean to enable server-side encryption (SSE) of message content with SQS-owned encryption keys | `bool` | `true` | no |
Expand Down
7 changes: 7 additions & 0 deletions modules/karpenter/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,13 @@ resource "aws_iam_role_policy_attachment" "irsa" {
policy_arn = aws_iam_policy.irsa[0].arn
}

resource "aws_iam_role_policy_attachment" "irsa_additional" {
for_each = { for k, v in var.policies : k => v if local.create_irsa }

role = aws_iam_role.irsa[0].name
policy_arn = each.value
}

################################################################################
# Node Termination Queue
################################################################################
Expand Down
6 changes: 6 additions & 0 deletions modules/karpenter/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,12 @@ variable "irsa_tags" {
default = {}
}

variable "policies" {
description = "Policies to attach to the IAM role in `{'static_name' = 'policy_arn'}` format"
type = map(string)
default = {}
}

variable "irsa_tag_key" {
description = "Tag key (`{key = value}`) applied to resources launched by Karpenter through the Karpenter provisioner"
type = string
Expand Down