Skip to content

claranet/terraform-azurerm-aks-light

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Azure Kubernetes Service

Changelog Notice Apache V2 License OpenTofu Registry

This Terraform module creates an Azure Kubernetes Service.

Non-exhaustive feature list, most of them can be overridden:

Why replacing the previous Claranet AKS module

This modules supersedes the previous AKS module. We've built a new module to clean up the technical debt that piled up due to fast-moving AKS product and keep backwards compatibility for existing users. Also, we've decided to remove all Kubernetes resources from this new module as a recommended best practice and for better tooling responsibility segregation.

So this module does less, that why light, but it should do it better.

Global versioning rule for Claranet Azure modules

Module version Terraform version OpenTofu version AzureRM version
>= 8.x.x Unverified 1.8.x >= 4.0
>= 7.x.x 1.3.x >= 3.0
>= 6.x.x 1.x >= 3.0
>= 5.x.x 0.15.x >= 2.0
>= 4.x.x 0.13.x / 0.14.x >= 2.0
>= 3.x.x 0.12.x >= 2.0
>= 2.x.x 0.12.x < 2.0
< 2.x.x 0.11.x < 2.0

Contributing

If you want to contribute to this repository, feel free to use our pre-commit git hook configuration which will help you automatically update and format some files for you by enforcing our Terraform code module best-practices.

More details are available in the CONTRIBUTING.md file.

Usage

This module is optimized to work with the Claranet terraform-wrapper tool which set some terraform variables in the environment needed by this module. More details about variables set by the terraform-wrapper available in the documentation.

⚠️ Since modules version v8.0.0, we do not maintain/check anymore the compatibility with Hashicorp Terraform. Instead, we recommend to use OpenTofu.

module "acr" {
  source  = "claranet/acr/azurerm"
  version = "x.x.x"

  location            = module.azure_region.location
  location_short      = module.azure_region.location_short
  client_name         = var.client_name
  environment         = var.environment
  stack               = var.stack
  resource_group_name = module.rg.name

  sku = "Standard"

  logs_destinations_ids = [module.run.log_analytics_workspace_id]
}

module "vnet" {
  source  = "claranet/vnet/azurerm"
  version = "x.x.x"

  location            = module.azure_region.location
  location_short      = module.azure_region.location_short
  client_name         = var.client_name
  environment         = var.environment
  stack               = var.stack
  resource_group_name = module.rg.name

  cidrs = ["10.0.0.0/19"]
}

module "nodes_subnet" {
  source  = "claranet/subnet/azurerm"
  version = "x.x.x"

  location_short      = module.azure_region.location_short
  client_name         = var.client_name
  environment         = var.environment
  stack               = var.stack
  resource_group_name = module.rg.name

  name_suffix = "nodes"

  virtual_network_name = module.vnet.name

  cidrs             = ["10.0.0.0/20"]
  service_endpoints = ["Microsoft.Storage", "Microsoft.KeyVault"]
}

module "private_dns_zone" {
  source  = "claranet/private-endpoint/azurerm//modules/private-dns-zone"
  version = "x.x.x"

  environment         = var.environment
  stack               = var.stack
  resource_group_name = module.rg.name

  name                = "privatelink.${module.azure_region.location_cli}.azmk8s.io"
  virtual_network_ids = [module.vnet.id]
}

resource "tls_private_key" "main" {
  algorithm = "RSA"
}

module "containers_logs" {
  source  = "claranet/run/azurerm//modules/logs"
  version = "x.x.x"

  client_name         = var.client_name
  environment         = var.environment
  location            = module.azure_region.location
  location_short      = module.azure_region.location_short
  stack               = var.stack
  resource_group_name = module.rg.name

  storage_account_enabled = true
  workspace_custom_name   = "log-aks-containers-${var.environment}-${module.azure_region.location_short}"
}

module "aks" {
  source  = "claranet/aks-light/azurerm"
  version = "x.x.x"

  location            = module.azure_region.location
  location_short      = module.azure_region.location_short
  client_name         = var.client_name
  environment         = var.environment
  stack               = var.stack
  resource_group_name = module.rg.name

  kubernetes_version = "1.30.4"
  service_cidr       = "10.0.16.0/22"

  nodes_subnet = {
    name                 = module.nodes_subnet.name
    virtual_network_name = module.vnet.name
  }

  private_cluster_enabled = true
  private_dns_zone_type   = "Custom"
  private_dns_zone_id     = module.private_dns_zone.id

  default_node_pool = {
    vm_size         = "Standard_B4ms"
    os_disk_size_gb = 64
  }

  node_pools = [
    {
      name                = "nodepool1"
      vm_size             = "Standard_B4ms"
      os_disk_type        = "Ephemeral"
      os_disk_size_gb     = 100
      vnet_subnet_id      = module.nodes_subnet.id
      enable_auto_scaling = true
      min_count           = 1
      max_count           = 10
    }
  ]

  linux_profile = {
    username = "nodeadmin"
    ssh_key  = tls_private_key.main.public_key_openssh
  }

  container_registry_id = module.acr.id

  oms_agent = {
    log_analytics_workspace_id = module.run.log_analytics_workspace_id
  }

  data_collection_rule = {
    custom_log_analytics_workspace_id = module.containers_logs.id
  }

  maintenance_window = {
    allowed = [{
      day   = "Monday"
      hours = [10, 11, 12, 13, 14]
    }]
  }

  maintenance_window_auto_upgrade = {
    frequency   = "RelativeMonthly"
    interval    = 1
    duration    = 4
    week_index  = "First"
    day_of_week = "Monday"
    start_time  = "10:00"
    utc_offset  = "+02:00"
  }

  logs_destinations_ids = [module.run.log_analytics_workspace_id]
}

Providers

Name Version
azapi ~> 2.0
azuread ~> 3.0
azurecaf ~> 1.2.28
azurerm ~> 4.0
null >= 3.0

Modules

Name Source Version
diagnostics claranet/diagnostic-settings/azurerm ~> 8.0.0

Resources

Name Type
azapi_update_resource.aks_kubernetes_version resource
azurerm_kubernetes_cluster.main resource
azurerm_kubernetes_cluster_node_pool.main resource
azurerm_monitor_data_collection_rule.main resource
azurerm_monitor_data_collection_rule_association.main resource
azurerm_role_assignment.aci_assignment resource
azurerm_role_assignment.kubelet_uai_acr_pull resource
azurerm_role_assignment.kubelet_uai_nodes_rg_contributor resource
azurerm_role_assignment.uai_private_dns_zone_contributor resource
azurerm_role_assignment.uai_route_table_contributor resource
azurerm_role_assignment.uai_subnets_network_contributor resource
azurerm_user_assigned_identity.main resource
null_resource.kubernetes_version_keeper resource
azapi_resource.subnet_delegation data source
azuread_service_principal.aci_identity data source
azurecaf_name.aks data source
azurecaf_name.aks_identity data source
azurecaf_name.aks_nodes_rg data source
azurecaf_name.dcr data source
azurerm_kubernetes_service_versions.main data source
azurerm_subscription.current data source

Inputs

Name Description Type Default Required
aci_subnet ID of the Subnet for ACI virtual-nodes.
object({
id = string
})
null no
aks_user_assigned_identity_custom_name Custom name for the AKS user assigned identity resource. string null no
api_server_authorized_ip_ranges IP ranges allowed to interact with Kubernetes API for public clusters.
See documentation about "0.0.0.0/32" default value :
- https://learn.microsoft.com/en-us/azure/aks/api-server-authorized-ip-ranges#allow-only-the-outbound-public-ip-of-the-standard-sku-load-balancer
- https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster#public_network_access_enabled

Set to 0.0.0.0/0 to wide open (not recommended)
list(string)
[
"0.0.0.0/32"
]
no
auto_scaler_profile Auto Scaler configuration.
object({
balance_similar_node_groups = optional(bool, false)
expander = optional(string, "random")
max_graceful_termination_sec = optional(number, 600)
max_node_provisioning_time = optional(string, "15m")
max_unready_nodes = optional(number, 3)
max_unready_percentage = optional(number, 45)
new_pod_scale_up_delay = optional(string, "10s")
scale_down_delay_after_add = optional(string, "10m")
scale_down_delay_after_delete = optional(string, "10s")
scale_down_delay_after_failure = optional(string, "3m")
scan_interval = optional(string, "10s")
scale_down_unneeded = optional(string, "10m")
scale_down_unready = optional(string, "20m")
scale_down_utilization_threshold = optional(number, 0.5)
empty_bulk_delete_max = optional(number, 10)
skip_nodes_with_local_storage = optional(bool, true)
skip_nodes_with_system_pods = optional(bool, true)
})
null no
automatic_upgrade_channel The upgrade channel for this Kubernetes Cluster. Possible values are patch, rapid, node-image and stable. Setting this field to null sets this value to none. string "patch" no
azure_active_directory_rbac Active Directory role based access control configuration.
object({
tenant_id = optional(string, null)
admin_group_object_ids = optional(list(string), [])
azure_rbac_enabled = optional(bool, true)
})
null no
azure_policy_enabled Option to enable Azure Policy add-on. bool true no
client_name Client name/account used in naming. string n/a yes
container_registry_id Azure Container Registry ID where Azure Kubernetes Service needs pull access. string null no
cost_analysis_enabled Option to enable cost analysis in the Azure portal for this Kubernetes cluster. The sku_tier must be set to Standard or Premium to enable this feature. bool false no
custom_name Custom AKS name, generated if not set. string "" no
data_collection_rule AKS Data Collection Rule configuration.
object({
enabled = optional(bool, true)
custom_log_analytics_workspace_id = optional(string)
data_streams = optional(list(string), [
"Microsoft-ContainerLog",
"Microsoft-ContainerLogV2",
"Microsoft-KubeEvents",
"Microsoft-KubePodInventory",
"Microsoft-InsightsMetrics",
"Microsoft-ContainerInventory",
"Microsoft-ContainerNodeInventory",
"Microsoft-KubeNodeInventory",
"Microsoft-KubeServices",
"Microsoft-KubePVInventory"
])
namespaces_filter = optional(list(string), [
"kube-system",
"gatekeeper-system",
"kube-node-lease",
"calico-system",
])
namespace_filtering_mode = optional(string, "Exclude")
data_collection_interval = optional(string, "5m")
container_log_v2_enabled = optional(bool, true)
})
{} no
data_collection_rule_custom_name Custom name for the AKS Data Collection Rule. string null no
default_node_pool Default Node Pool configuration.
object({
name = optional(string, "default")
type = optional(string, "VirtualMachineScaleSets")
vm_size = optional(string, "Standard_D2_v3")
os_sku = optional(string, "Ubuntu")
os_disk_type = optional(string, "Managed")
os_disk_size_gb = optional(number)
auto_scaling_enabled = optional(bool, false)
node_count = optional(number, 1)
min_count = optional(number, 1)
max_count = optional(number, 10)
max_pods = optional(number)
node_labels = optional(map(any))
node_taints = optional(list(any))
host_encryption_enabled = optional(bool)
node_public_ip_enabled = optional(bool, false)
orchestrator_version = optional(string)
zones = optional(list(number), [1, 2, 3])
tags = optional(map(string), {})
temporary_name_for_rotation = optional(string)
upgrade_settings = optional(object({
max_surge = optional(string, "10%")
}), {})
linux_os_config = optional(object({
swap_file_size_mb = optional(number)
transparent_huge_page_enabled = optional(string)
transparent_huge_page_defrag = optional(string)
sysctl_config = optional(object({
fs_aio_max_nr = optional(number)
fs_file_max = optional(number)
fs_inotify_max_user_watches = optional(number)
fs_nr_open = optional(number)
kernel_threads_max = optional(number)
net_core_netdev_max_backlog = optional(number)
net_core_optmem_max = optional(number)
net_core_rmem_default = optional(number)
net_core_rmem_max = optional(number)
net_core_somaxconn = optional(number)
net_core_wmem_default = optional(number)
net_core_wmem_max = optional(number)
net_ipv4_ip_local_port_range_max = optional(number)
net_ipv4_ip_local_port_range_min = optional(number)
net_ipv4_neigh_default_gc_thresh1 = optional(number)
net_ipv4_neigh_default_gc_thresh2 = optional(number)
net_ipv4_neigh_default_gc_thresh3 = optional(number)
net_ipv4_tcp_fin_timeout = optional(number)
net_ipv4_tcp_keepalive_intvl = optional(number)
net_ipv4_tcp_keepalive_probes = optional(number)
net_ipv4_tcp_keepalive_time = optional(number)
net_ipv4_tcp_max_syn_backlog = optional(number)
net_ipv4_tcp_max_tw_buckets = optional(number)
net_ipv4_tcp_tw_reuse = optional(bool)
net_netfilter_nf_conntrack_buckets = optional(number)
net_netfilter_nf_conntrack_max = optional(number)
vm_max_map_count = optional(number)
vm_swappiness = optional(number)
vm_vfs_cache_pressure = optional(number)
}))
}))
})
{} no
default_tags_enabled Option to enable or disable default tags. bool true no
diagnostic_settings_custom_name Custom name of the diagnostics settings, name will be 'default' if not set. string "default" no
environment Project environment. string n/a yes
extra_tags Additional tags to add on resources. map(string) {} no
http_application_routing_enabled Whether HTTP Application Routing is enabled. bool false no
http_proxy_settings Azure Kubernetes Service HTTP proxy settings. URLs must be in format http(s)://fqdn:port/. When setting the no_proxy_list parameter, the AKS Private Endpoint domain name and the AKS VNet CIDR (or Subnet CIDRs) must be added to the list.
object({
https_proxy_url = optional(string)
http_proxy_url = optional(string)
trusted_ca = optional(string)
no_proxy_list = optional(list(string), [])
})
null no
image_cleaner_configuration Kubernetes image cleaner configuration.
object({
enabled = optional(bool, true)
interval_hours = optional(number, 24)
})
{} no
key_vault_secrets_provider Enable AKS built-in Key Vault secrets provider. If enabled, an identity is created by the AKS itself and exported from this module.
object({
secret_rotation_enabled = optional(bool, true)
secret_rotation_interval = optional(string)
})
{} no
kubernetes_version Version of Kubernetes to deploy. string null no
linux_profile Username and SSH public key for accessing Linux nodes with SSH.
object({
username = string
ssh_key = string
})
null no
location Azure region to use. string n/a yes
location_short Short string for Azure location. string n/a yes
logs_categories Log categories to send to destinations. list(string) null no
logs_destinations_ids List of destination resources IDs for logs diagnostic destination.
Can be Storage Account, Log Analytics Workspace and Event Hub. No more than one of each can be set.
If you want to use Azure EventHub as a destination, you must provide a formatted string containing both the EventHub Namespace authorization send ID and the EventHub name (name of the queue to use in the Namespace) separated by the | character.
list(string) n/a yes
logs_kube_audit_enabled Whether to include kube-audit and kube-audit-admin logs from diagnostics settings collection. Enabling this can increase your Azure billing. bool false no
logs_metrics_categories Metrics categories to send to destinations. list(string) null no
maintenance_window Maintenance window configuration. This is the basic configuration for controlling AKS releases. See documentation.
object({
allowed = optional(list(object({
day = string
hours = list(number)
})), [])
not_allowed = optional(list(object({
start = string
end = string
})), [])
})
null no
maintenance_window_auto_upgrade Controls when to perform cluster upgrade with more finely controlled cadence and recurrence settings compared to the basic one. See documentation.
object({
frequency = string
interval = string
duration = number
day_of_week = optional(string)
day_of_month = optional(string)
week_index = optional(string)
start_time = string
utc_offset = optional(string)
start_date = optional(string)
not_allowed = optional(list(object({
start = string
end = string
})), [])
})
null no
monitor_metrics Specifies a Prometheus add-on profile for this Kubernetes Cluster.
object({
annotations_allowed = optional(string, null)
labels_allowed = optional(string, null)
})
null no
name_prefix Optional prefix for the generated name. string "" no
name_suffix Optional suffix for the generated name. string "" no
network_mode Azure Kubernetes Service network mode to use. Only available with Azure CNI. string null no
network_plugin Azure Kubernetes Service network plugin to use. Possible names are azure and kubenet. Possible CNI modes are None, Overlay and Cilium for Azure CNI and None for Kubenet. Changing this forces a new resource to be created.
object({
name = optional(string, "azure")
cni_mode = optional(string, "overlay")
})
{} no
network_policy Azure Kubernetes Service network policy to use. string "calico" no
node_pools A list of Node Pools to create.
list(object({
name = string
vm_size = optional(string, "Standard_D2_v3")
os_sku = optional(string, "Ubuntu")
os_disk_type = optional(string, "Managed")
os_disk_size_gb = optional(number)
kubelet_disk_type = optional(string)
linux_os_config = optional(object({
swap_file_size_mb = optional(number)
transparent_huge_page_enabled = optional(string)
transparent_huge_page_defrag = optional(string)
sysctl_config = optional(object({
fs_aio_max_nr = optional(number)
fs_file_max = optional(number)
fs_inotify_max_user_watches = optional(number)
fs_nr_open = optional(number)
kernel_threads_max = optional(number)
net_core_netdev_max_backlog = optional(number)
net_core_optmem_max = optional(number)
net_core_rmem_default = optional(number)
net_core_rmem_max = optional(number)
net_core_somaxconn = optional(number)
net_core_wmem_default = optional(number)
net_core_wmem_max = optional(number)
net_ipv4_ip_local_port_range_max = optional(number)
net_ipv4_ip_local_port_range_min = optional(number)
net_ipv4_neigh_default_gc_thresh1 = optional(number)
net_ipv4_neigh_default_gc_thresh2 = optional(number)
net_ipv4_neigh_default_gc_thresh3 = optional(number)
net_ipv4_tcp_fin_timeout = optional(number)
net_ipv4_tcp_keepalive_intvl = optional(number)
net_ipv4_tcp_keepalive_probes = optional(number)
net_ipv4_tcp_keepalive_time = optional(number)
net_ipv4_tcp_max_syn_backlog = optional(number)
net_ipv4_tcp_max_tw_buckets = optional(number)
net_ipv4_tcp_tw_reuse = optional(bool)
net_netfilter_nf_conntrack_buckets = optional(number)
net_netfilter_nf_conntrack_max = optional(number)
vm_max_map_count = optional(number)
vm_swappiness = optional(number)
vm_vfs_cache_pressure = optional(number)
}))
}))
auto_scaling_enabled = optional(bool, false)
node_count = optional(number, 1)
min_count = optional(number, 1)
max_count = optional(number, 10)
max_pods = optional(number)
node_labels = optional(map(any))
node_taints = optional(list(any))
host_encryption_enabled = optional(bool)
node_public_ip_enabled = optional(bool, false)
node_subnet = optional(object({
name = optional(string)
virtual_network_name = optional(string)
resource_group_name = optional(string)
}), {})
pod_subnet = optional(object({
name = optional(string)
virtual_network_name = optional(string)
resource_group_name = optional(string)
}), {})
priority = optional(string)
eviction_policy = optional(string)
orchestrator_version = optional(string)
upgrade_settings = optional(object({
max_surge = optional(string, "10%")
}), {})
zones = optional(list(number), [1, 2, 3])
tags = optional(map(string), {})
}))
[] no
nodes_resource_group_name Name of the Resource Group in which to put Azure Kubernetes Service nodes. string null no
nodes_subnet The Subnet used by nodes.
object({
name = string
virtual_network_name = string
resource_group_name = optional(string)
})
n/a yes
oidc_issuer_enabled Whether the OIDC issuer URL should be enabled. bool true no
oms_agent OMS Agent configuration.
object({
log_analytics_workspace_id = optional(string)
msi_auth_for_monitoring_enabled = optional(bool, true)
})
n/a yes
outbound_type The outbound (egress) routing method which should be used. Possible values are loadBalancer and userDefinedRouting. string "loadBalancer" no
pod_cidr CIDR used by pods when network plugin is set to kubenet or azure CNI Overlay. string null no
pods_subnet The Subnet containing the pods.
object({
name = optional(string)
virtual_network_name = optional(string)
resource_group_name = optional(string)
})
{} no
private_cluster_enabled Configure Azure Kubernetes Service as a Private Cluster. See documentation. bool true no
private_cluster_public_fqdn_enabled Specifies whether a Public FQDN for this Private Cluster should be added. See documentation. bool false no
private_dns_zone_id ID of the Private DNS Zone when private_dns_zone_type = "Custom". string null no
private_dns_zone_role_assignment_enabled Option to enable or disable Private DNS Zone role assignment. bool true no
private_dns_zone_type Set Azure Kubernetes Service private DNS zone if needed and if private cluster is enabled (privatelink..azmk8s.io)
- "Custom" : You will have to deploy a private DNS Zone on your own and provide the ID with <private_dns_zone_id> variable
- "System" : AKS will manage the Private DNS Zone and creates it in the Node Resource Group
- "None" : In case of None you will need to bring your own DNS server and set up resolving, otherwise cluster will have issues after provisioning.

See documentation.
string "System" no
resource_group_name Name of the resource group. string n/a yes
route_table_id Provide an existing Route Table ID when outbound_type = "userDefinedRouting". Only available with Kubenet. string null no
service_cidr CIDR used by Kubernetes services (kubectl get svc). string n/a yes
sku_tier Azure Kubernetes Service SKU tier. Possible values are Free ou Standard. string "Standard" no
stack Project stack name. string n/a yes
storage_profile Select the CSI drivers to be enabled.
object({
blob_driver_enabled = optional(bool, true)
disk_driver_enabled = optional(bool, true)
file_driver_enabled = optional(bool, true)
snapshot_controller_enabled = optional(bool, true)
})
null no
user_assigned_identity_resource_group_name Resource Group where to deploy the Azure Kubernetes Service User Assigned Identity resource. string null no
user_assigned_identity_tags Tags to add to AKS MSI. map(string) {} no
workload_identity_enabled Whether Azure AD Workload Identity should be enabled for the cluster. oidc_issuer_enabled must be set to true to use this feature. bool true no

Outputs

Name Description
apiserver_endpoint APIServer Endpoint of the Azure Kubernetes Service.
id ID of the Azure Kubernetes Service.
identity_principal_id AKS System Managed Identity principal ID.
key_vault_secrets_provider_identity The User Managed Identity used by the Key Vault secrets provider.
kube_config Kube configuration of the Azure Kubernetes Service.
kube_config_raw Raw kubeconfig to be used by kubectl command.
kubelet_user_managed_identity The Kubelet User Managed Identity used by the Azure Kubernetes Service.
kubernetes_version Azure Kubernetes Service Kubernetes version.
managed_private_dns_zone_id ID of the AKS' managed Private DNS Zone.
managed_private_dns_zone_name Name of the AKS' managed Private DNS Zone.
managed_private_dns_zone_resource_group_name Resource Group name of the AKS' managed Private DNS Zone.
module_diagnostics Diagnostics Settings module output.
name Name of the Azure Kubernetes Service.
node_pools Map of Azure Kubernetes Service Node Pools attributes.
nodes_resource_group_name Name of the Resource Group in which Azure Kubernetes Service nodes are deployed.
oidc_issuer_url The OIDC issuer URL that is associated with the Azure Kubernetes Service.
portal_fqdn Portal FQDN of the Azure Kubernetes Service.
private_cluster_enabled Whether private cluster is enabled.
private_fqdn Private FQDNs of the Azure Kubernetes Service.
public_fqdn Public FQDN of the Azure Kubernetes Service.
resource Azure Kubernetes Cluster resource object.
resource_data_collection_rule Data Collection Rule resource output.
resource_node_pools Azure Kubernetes Node Pools resource output.
user_managed_identity The User Managed Identity used by the Azure Kubernetes Service.

Related documentation