From 333b99f4fc6b2b9bdbe21369e3f239982cf962e1 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Fri, 1 Mar 2019 16:11:51 -0800 Subject: [PATCH 01/16] Documentation for Windows GMSA feature Signed-off-by: Deep Debroy --- .../getting-started-guides/windows/_index.md | 274 ++++++++++++++++++ 1 file changed, 274 insertions(+) diff --git a/content/en/docs/getting-started-guides/windows/_index.md b/content/en/docs/getting-started-guides/windows/_index.md index 5d0ca0d122804..f7caad612278d 100644 --- a/content/en/docs/getting-started-guides/windows/_index.md +++ b/content/en/docs/getting-started-guides/windows/_index.md @@ -297,6 +297,7 @@ This example assumes you are running on Windows Server 1709, so uses the image t ### Secrets and ConfigMaps Secrets and ConfigMaps can be utilized in Windows Server Containers, but must be used as environment variables. See limitations section below for additional details. + **Examples:** Windows pod with secrets mapped to environment variables @@ -378,6 +379,279 @@ PS > Get-Service kubelet; Get-Service kube-proxy; CMD > sc.exe queryex kubelet && sc qc kubelet && sc.exe queryex kube-proxy && sc.exe qc kube-proxy ``` +### Group Managed Service Accounts (GMSA) support + +{{< feature-state for_k8s_version="v1.14" state="alpha" >}} + +Starting with Kubernetes v1.14, Windows container workloads can be configured to use Group Managed Service Accounts (GMSA). Group Managed Service Accounts are a specific type of Active Directory account that provides automatic password management, simplified service principal name (SPN) management, and the ability to delegate the management to other administrators across multiple servers. For more information regarding GMSA, please see: https://docs.microsoft.com/en-us/windows-server/security/group-managed-service-accounts/group-managed-service-accounts-overview + +In Kubernetes, GMSA credential specs are configured at a Kubernetes cluster-wide scope as custom resources. Windows pods, as well as individual containers within a pod, can be configured to use a GMSA for domain based functions (e.g. Kerberos authentication) when interacting with other Windows services. As of v1.14, the only container runtime interface that supports GMSA for Windows workloads is Dockershim. Implementation of GMSA through CRI and other runtimes is planned for the future. + +#### Setup +As an alpha feature, there are some additional setup steps that must be taken in order to use the GMSA feature: +1. Enable the `WindowsGMSA` feature gate on kubelet on Windows nodes. +2. Install the GMSACredentialSpec CRD +3. Create GMSA credential spec resources +4. Install webhooks to expand and validate references to GMSA credential spec resources from pod specs +5. Create cluster roles to allow service accounts to use GMSA credential spec resources +6. Configure pods to use GMSA credential specs along with a service account authorized to use the GMSA credential specs + +##### Enable the WindowsGMSA feature gate +In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. + +##### Install the GMSACredentialSpec CRD +The following [CustomResourceDefinition][] (CRD) for GMSA credential spec resources needs to be configured on the cluster: + +``` +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: gmsacredentialspecs.windows.k8s.io +spec: + group: windows.k8s.io + version: v1alpha1 + names: + kind: GMSACredentialSpec + plural: gmsacredentialspecs + scope: Cluster + validation: + openAPIV3Schema: + properties: + credspec: + description: GMSA Credential Spec + type: object +``` + +Install the CRD with `kubectl apply -f gmsa-crd.yml` + +[CustomResourceDefinition]: /docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/ + +##### Create GMSA credspec resources +With the GMSACredentialSpec CRD installed, the corresponding GMSA credspec custom resources can be configured. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Steps for using it and generating a GMSA credspec YAML based on the JSON: +1. Import the CredentialSpec module: `ipmo CredentialSpec.psm1` +2. Create a credential spec in JSON format using `New-CredentialSpec`. Sample invocation for GMSA named WebApp1: `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` +3. Use `Get-CredentialSpec` to show the path of the JSON file. +4. Convert the cred spec file from JSON to YAML format and apply the necessary heading. An example is below. + +Note that the GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. + +``` +apiVersion: windows.k8s.io/v1alpha1 +kind: GMSACredentialSpec +metadata: + name: gmsa-WebApp1 #This is an arbitrary name but it will be used as a reference +credspec: + ActiveDirectoryConfig: + GroupManagedServiceAccounts: + - Name: WebApp1 #Username of the GMSA account + Scope: CONTOSO #NETBIOS Domain Name + - Name: WebApp1 #Username of the GMSA account + Scope: contoso.com #DNS Domain Name + CmsPlugins: + - ActiveDirectory + DomainJoinConfig: + DnsName: contoso.com #DNS Domain Name + DnsTreeName: contoso.com #DNS Domain Name Root + Guid: 244818ae-87ac-4fcd-92ec-e79e5252348a #GUID + MachineAccountName: WebApp1 #Username of the GMSA account + NetBiosName: CONTOSO #NETBIOS Domain Name + Sid: S-1-5-21-2126449477-2524075714-3094792973 #SID of GMSA +``` + +5. Deploy the credential spec resource: `kubectl apply -f gmsa-Webapp1-credspec.yml` + +[PowerShell]: https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1 + +#### Install Webhooks for GMSA +Two webhooks need to be configured on the Kubernetes cluster to populate and validate GMSA credential spec references at the pod or container level. The mutating webhook expands references to GMSAs (by name from a pod specification) into the full credential spec in JSON form within the pod spec. The validating webhook ensures all references to GMSAs are authorized to be used by the pod service account. + +Installing the webhooks require several steps: First, create a certificate key pair (that will be used to allow the webhook container to communicate to the cluster) and install a corresponding secret. Next, create a deployment for the core webhook logic. Finally, create the validating and mutating webhook configurations referring to the deployment. The YAML file below can be used to configure the webhooks and the associated deployment and secret. Replace the values in curly braces {} with desired values, save it as `gmsa-webhooks.yml` and apply using `kubectl apply -f gmsa-webhooks.yml` + +``` +apiVersion: v1 +kind: Secret +metadata: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} +data: + tls_private_key: ${TLS_PRIVATE_KEY} + tls_certificate: ${TLS_CERTIFICATE} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} +spec: + replicas: 1 + selector: + matchLabels: + app: ${DEPLOYMENT_NAME} + template: + metadata: + labels: + app: ${DEPLOYMENT_NAME} + spec: + containers: + - name: ${DEPLOYMENT_NAME} + image: k8ssigwindows/gmsa-admission-webhook + imagePullPolicy: IfNotPresent + ports: + - containerPort: 443 + volumeMounts: + - name: tls + mountPath: "/tls" + readOnly: true + env: + - name: TLS_KEY + value: /tls/key + - name: TLS_CRT + value: /tls/crt + volumes: + - name: tls + secret: + secretName: ${DEPLOYMENT_NAME} + items: + - key: tls_private_key + path: key + - key: tls_certificate + path: crt +--- +apiVersion: v1 +kind: Service +metadata: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} +spec: + ports: + - port: 443 + targetPort: 443 + selector: + app: ${DEPLOYMENT_NAME} +--- +# add a label to the deployment's namespace so that we can exclude it +apiVersion: v1 +kind: Namespace +metadata: + name: ${NAMESPACE} + labels: + gmsa-webhook: disabled +--- +apiVersion: admissionregistration.k8s.io/v1beta1 +kind: ValidatingWebhookConfiguration +metadata: + name: ${DEPLOYMENT_NAME} +webhooks: +- name: k8s-gmsa-admission-webhook.sig-windows.k8s.io + clientConfig: + service: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} + path: "/validate" + caBundle: ${CA_BUNDLE} + rules: + - operations: ["CREATE", "UPDATE"] + apiGroups: [""] + apiVersions: ["*"] + resources: ["pods"] + failurePolicy: Fail + # don't run on ${NAMESPACE} + namespaceSelector: + matchExpressions: + - key: gmsa-webhook + operator: NotIn + values: [disabled] +--- +apiVersion: admissionregistration.k8s.io/v1beta1 +kind: MutatingWebhookConfiguration +metadata: + name: ${DEPLOYMENT_NAME} +webhooks: +- name: k8s-gmsa-admission-webhook.sig-windows.k8s.io + clientConfig: + service: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} + path: "/mutate" + caBundle: ${CA_BUNDLE} + rules: + - operations: ["CREATE"] + apiGroups: [""] + apiVersions: ["*"] + resources: ["pods"] + failurePolicy: Fail + # don't run on ${NAMESPACE} + namespaceSelector: + matchExpressions: + - key: gmsa-webhook + operator: NotIn + values: [disabled] +``` + +##### Configure cluster role and RBAC for service accounts to use GMSA credential specs +A service account that a pod is configured with needs to be authorized for the "use" verb on the desired GMSA credential spec resources. The following is a template using the default service account demonstrating the cluster role and role binding to use a GMSA credential spec from above. Save the file as gmsa-roles.yml and apply using `kubectl apply -f gmsa-roles.yml` + +``` +#Create the Role to read the credspec +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: my-rbac-reader +rules: +- apiGroups: ["windows.k8s.io"] + resources: ["gmsacredentialspecs"] + verbs: ["use"] + resourceNames: ["gmsa-WebApp1"] +--- +#Assign the role to the service account +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: allow-default-svc-account-read-on-gmsa-WebApp1 + namespace: default +subjects: +- kind: ServiceAccount + name: default + namespace: default +roleRef: + kind: ClusterRole + name: my-rbac-reader + apiGroup: rbac.authorization.k8s.io +``` + +##### Configure GMSA credential spec reference in pod spec +In the alpha stage of the feature, the annotation `pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name` is used to specify references to desired GMSA credential spec custom resources from pod specs. A sample pod spec with the annotation populated: + +``` +apiVersion: apps/v1beta1 +kind: Deployment +metadata: + labels: + run: with-creds + name: with-creds + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + run: with-creds + template: + metadata: + creationTimestamp: null + labels: + run: with-creds + annotations: + pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 + spec: + containers: + - image: xxxxx/test-ad-loop + imagePullPolicy: Always + name: with-creds + nodeSelector: + beta.kubernetes.io/os: windows +``` + + ## Known Limitations for Windows Server Containers with v1.9 Some of these limitations will be addressed by the community in future releases of Kubernetes: From ed528f8380c629573590a6b33099e7fb52ff68e4 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Mon, 11 Mar 2019 09:49:40 -0700 Subject: [PATCH 02/16] Enhancements to GMSA docs Signed-off-by: Deep Debroy --- .../getting-started-guides/windows/_index.md | 35 ++++--------------- 1 file changed, 6 insertions(+), 29 deletions(-) diff --git a/content/en/docs/getting-started-guides/windows/_index.md b/content/en/docs/getting-started-guides/windows/_index.md index f7caad612278d..fb12990c7ae7e 100644 --- a/content/en/docs/getting-started-guides/windows/_index.md +++ b/content/en/docs/getting-started-guides/windows/_index.md @@ -397,43 +397,20 @@ As an alpha feature, there are some additional setup steps that must be taken in 6. Configure pods to use GMSA credential specs along with a service account authorized to use the GMSA credential specs ##### Enable the WindowsGMSA feature gate -In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. +In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. See [Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling feature gates. ##### Install the GMSACredentialSpec CRD -The following [CustomResourceDefinition][] (CRD) for GMSA credential spec resources needs to be configured on the cluster: - -``` -apiVersion: apiextensions.k8s.io/v1beta1 -kind: CustomResourceDefinition -metadata: - name: gmsacredentialspecs.windows.k8s.io -spec: - group: windows.k8s.io - version: v1alpha1 - names: - kind: GMSACredentialSpec - plural: gmsacredentialspecs - scope: Cluster - validation: - openAPIV3Schema: - properties: - credspec: - description: GMSA Credential Spec - type: object -``` - -Install the CRD with `kubectl apply -f gmsa-crd.yml` +A [CustomResourceDefinition](CRD) for GMSA credential spec resources needs to be configured on the cluster to define the custom resource type `GMSACredentialSpec`. Download the GMSA CRD [YAML](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl#L131-L148) and save it as gmsa-crd.yaml. +Next, install the CRD with `kubectl apply -f gmsa-crd.yaml` [CustomResourceDefinition]: /docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/ ##### Create GMSA credspec resources -With the GMSACredentialSpec CRD installed, the corresponding GMSA credspec custom resources can be configured. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Steps for using it and generating a GMSA credspec YAML based on the JSON: +With the GMSACredentialSpec CRD installed, GMSA credspec custom resources can now be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Following are the steps for generating a GMSA credspec YAML based on the JSON: 1. Import the CredentialSpec module: `ipmo CredentialSpec.psm1` -2. Create a credential spec in JSON format using `New-CredentialSpec`. Sample invocation for GMSA named WebApp1: `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` +2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credspec named WebApp1: `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` 3. Use `Get-CredentialSpec` to show the path of the JSON file. -4. Convert the cred spec file from JSON to YAML format and apply the necessary heading. An example is below. - -Note that the GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. +4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource. An example based on a GMSA credspec named WebApp1 is below: ``` apiVersion: windows.k8s.io/v1alpha1 From e69f2220bfdf0f69e3b10cf6249608f244d0480f Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Mon, 11 Mar 2019 09:53:50 -0700 Subject: [PATCH 03/16] Fix links Signed-off-by: Deep Debroy --- content/en/docs/getting-started-guides/windows/_index.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/content/en/docs/getting-started-guides/windows/_index.md b/content/en/docs/getting-started-guides/windows/_index.md index fb12990c7ae7e..f9bf74e34b6f2 100644 --- a/content/en/docs/getting-started-guides/windows/_index.md +++ b/content/en/docs/getting-started-guides/windows/_index.md @@ -397,14 +397,12 @@ As an alpha feature, there are some additional setup steps that must be taken in 6. Configure pods to use GMSA credential specs along with a service account authorized to use the GMSA credential specs ##### Enable the WindowsGMSA feature gate -In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. See [Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling feature gates. +In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. See [Feature Gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling feature gates. ##### Install the GMSACredentialSpec CRD -A [CustomResourceDefinition](CRD) for GMSA credential spec resources needs to be configured on the cluster to define the custom resource type `GMSACredentialSpec`. Download the GMSA CRD [YAML](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl#L131-L148) and save it as gmsa-crd.yaml. +A [CustomResourceDefinition](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/) (CRD) for GMSA credential spec resources needs to be configured on the cluster to define the custom resource type `GMSACredentialSpec`. Download the GMSA CRD [YAML](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl#L131-L148) and save it as gmsa-crd.yaml. Next, install the CRD with `kubectl apply -f gmsa-crd.yaml` -[CustomResourceDefinition]: /docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/ - ##### Create GMSA credspec resources With the GMSACredentialSpec CRD installed, GMSA credspec custom resources can now be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Following are the steps for generating a GMSA credspec YAML based on the JSON: 1. Import the CredentialSpec module: `ipmo CredentialSpec.psm1` From ade2515cb7f3023dd3d4f22423407665bc6f2029 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Mon, 11 Mar 2019 09:55:18 -0700 Subject: [PATCH 04/16] Fix GMSA link Signed-off-by: Deep Debroy --- content/en/docs/getting-started-guides/windows/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/getting-started-guides/windows/_index.md b/content/en/docs/getting-started-guides/windows/_index.md index f9bf74e34b6f2..340693a8719a4 100644 --- a/content/en/docs/getting-started-guides/windows/_index.md +++ b/content/en/docs/getting-started-guides/windows/_index.md @@ -383,7 +383,7 @@ CMD > sc.exe queryex kubelet && sc qc kubelet && sc.exe queryex kube-proxy && sc {{< feature-state for_k8s_version="v1.14" state="alpha" >}} -Starting with Kubernetes v1.14, Windows container workloads can be configured to use Group Managed Service Accounts (GMSA). Group Managed Service Accounts are a specific type of Active Directory account that provides automatic password management, simplified service principal name (SPN) management, and the ability to delegate the management to other administrators across multiple servers. For more information regarding GMSA, please see: https://docs.microsoft.com/en-us/windows-server/security/group-managed-service-accounts/group-managed-service-accounts-overview +Starting with Kubernetes v1.14, Windows container workloads can be configured to use [Group Managed Service Accounts](https://docs.microsoft.com/en-us/windows-server/security/group-managed-service-accounts/group-managed-service-accounts-overview) (GMSA). Group Managed Service Accounts are a specific type of Active Directory account that provides automatic password management, simplified service principal name (SPN) management, and the ability to delegate the management to other administrators across multiple servers. In Kubernetes, GMSA credential specs are configured at a Kubernetes cluster-wide scope as custom resources. Windows pods, as well as individual containers within a pod, can be configured to use a GMSA for domain based functions (e.g. Kerberos authentication) when interacting with other Windows services. As of v1.14, the only container runtime interface that supports GMSA for Windows workloads is Dockershim. Implementation of GMSA through CRI and other runtimes is planned for the future. From cbe7529b7b08913ae03fbfdb6de832f35f07074a Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Mon, 11 Mar 2019 11:26:35 -0700 Subject: [PATCH 05/16] Add GMSA feature flag in feature flag list Signed-off-by: Deep Debroy --- .../reference/command-line-tools-reference/feature-gates.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index 43dc0c9be627c..e6bdf35c7dff0 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -147,6 +147,7 @@ different Kubernetes components. | `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | - | | `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 | | `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | | +| `WindowsGMSA` | `false` | Alpha | 1.14 | | ## Using a Feature @@ -311,5 +312,6 @@ Each feature gate is designed for enabling/disabling a specific feature: type when used together with the `PersistentLocalVolumes` feature gate. - `VolumeSnapshotDataSource`: Enable volume snapshot data source support. - `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment variables into a `subPath`. +- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. {{% /capture %}} From 80172ea35c8895c38ebd8cba5faac17502afefd0 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Tue, 12 Mar 2019 19:03:22 -0700 Subject: [PATCH 06/16] Relocate GMSA to container configuration Signed-off-by: Deep Debroy --- .../getting-started-guides/windows/_index.md | 248 ---------------- .../configure-pod-container/configure-gmsa.md | 273 ++++++++++++++++++ 2 files changed, 273 insertions(+), 248 deletions(-) create mode 100644 content/en/docs/tasks/configure-pod-container/configure-gmsa.md diff --git a/content/en/docs/getting-started-guides/windows/_index.md b/content/en/docs/getting-started-guides/windows/_index.md index 340693a8719a4..a136755e139ca 100644 --- a/content/en/docs/getting-started-guides/windows/_index.md +++ b/content/en/docs/getting-started-guides/windows/_index.md @@ -379,254 +379,6 @@ PS > Get-Service kubelet; Get-Service kube-proxy; CMD > sc.exe queryex kubelet && sc qc kubelet && sc.exe queryex kube-proxy && sc.exe qc kube-proxy ``` -### Group Managed Service Accounts (GMSA) support - -{{< feature-state for_k8s_version="v1.14" state="alpha" >}} - -Starting with Kubernetes v1.14, Windows container workloads can be configured to use [Group Managed Service Accounts](https://docs.microsoft.com/en-us/windows-server/security/group-managed-service-accounts/group-managed-service-accounts-overview) (GMSA). Group Managed Service Accounts are a specific type of Active Directory account that provides automatic password management, simplified service principal name (SPN) management, and the ability to delegate the management to other administrators across multiple servers. - -In Kubernetes, GMSA credential specs are configured at a Kubernetes cluster-wide scope as custom resources. Windows pods, as well as individual containers within a pod, can be configured to use a GMSA for domain based functions (e.g. Kerberos authentication) when interacting with other Windows services. As of v1.14, the only container runtime interface that supports GMSA for Windows workloads is Dockershim. Implementation of GMSA through CRI and other runtimes is planned for the future. - -#### Setup -As an alpha feature, there are some additional setup steps that must be taken in order to use the GMSA feature: -1. Enable the `WindowsGMSA` feature gate on kubelet on Windows nodes. -2. Install the GMSACredentialSpec CRD -3. Create GMSA credential spec resources -4. Install webhooks to expand and validate references to GMSA credential spec resources from pod specs -5. Create cluster roles to allow service accounts to use GMSA credential spec resources -6. Configure pods to use GMSA credential specs along with a service account authorized to use the GMSA credential specs - -##### Enable the WindowsGMSA feature gate -In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. See [Feature Gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling feature gates. - -##### Install the GMSACredentialSpec CRD -A [CustomResourceDefinition](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/) (CRD) for GMSA credential spec resources needs to be configured on the cluster to define the custom resource type `GMSACredentialSpec`. Download the GMSA CRD [YAML](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl#L131-L148) and save it as gmsa-crd.yaml. -Next, install the CRD with `kubectl apply -f gmsa-crd.yaml` - -##### Create GMSA credspec resources -With the GMSACredentialSpec CRD installed, GMSA credspec custom resources can now be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Following are the steps for generating a GMSA credspec YAML based on the JSON: -1. Import the CredentialSpec module: `ipmo CredentialSpec.psm1` -2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credspec named WebApp1: `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` -3. Use `Get-CredentialSpec` to show the path of the JSON file. -4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource. An example based on a GMSA credspec named WebApp1 is below: - -``` -apiVersion: windows.k8s.io/v1alpha1 -kind: GMSACredentialSpec -metadata: - name: gmsa-WebApp1 #This is an arbitrary name but it will be used as a reference -credspec: - ActiveDirectoryConfig: - GroupManagedServiceAccounts: - - Name: WebApp1 #Username of the GMSA account - Scope: CONTOSO #NETBIOS Domain Name - - Name: WebApp1 #Username of the GMSA account - Scope: contoso.com #DNS Domain Name - CmsPlugins: - - ActiveDirectory - DomainJoinConfig: - DnsName: contoso.com #DNS Domain Name - DnsTreeName: contoso.com #DNS Domain Name Root - Guid: 244818ae-87ac-4fcd-92ec-e79e5252348a #GUID - MachineAccountName: WebApp1 #Username of the GMSA account - NetBiosName: CONTOSO #NETBIOS Domain Name - Sid: S-1-5-21-2126449477-2524075714-3094792973 #SID of GMSA -``` - -5. Deploy the credential spec resource: `kubectl apply -f gmsa-Webapp1-credspec.yml` - -[PowerShell]: https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1 - -#### Install Webhooks for GMSA -Two webhooks need to be configured on the Kubernetes cluster to populate and validate GMSA credential spec references at the pod or container level. The mutating webhook expands references to GMSAs (by name from a pod specification) into the full credential spec in JSON form within the pod spec. The validating webhook ensures all references to GMSAs are authorized to be used by the pod service account. - -Installing the webhooks require several steps: First, create a certificate key pair (that will be used to allow the webhook container to communicate to the cluster) and install a corresponding secret. Next, create a deployment for the core webhook logic. Finally, create the validating and mutating webhook configurations referring to the deployment. The YAML file below can be used to configure the webhooks and the associated deployment and secret. Replace the values in curly braces {} with desired values, save it as `gmsa-webhooks.yml` and apply using `kubectl apply -f gmsa-webhooks.yml` - -``` -apiVersion: v1 -kind: Secret -metadata: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} -data: - tls_private_key: ${TLS_PRIVATE_KEY} - tls_certificate: ${TLS_CERTIFICATE} ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} -spec: - replicas: 1 - selector: - matchLabels: - app: ${DEPLOYMENT_NAME} - template: - metadata: - labels: - app: ${DEPLOYMENT_NAME} - spec: - containers: - - name: ${DEPLOYMENT_NAME} - image: k8ssigwindows/gmsa-admission-webhook - imagePullPolicy: IfNotPresent - ports: - - containerPort: 443 - volumeMounts: - - name: tls - mountPath: "/tls" - readOnly: true - env: - - name: TLS_KEY - value: /tls/key - - name: TLS_CRT - value: /tls/crt - volumes: - - name: tls - secret: - secretName: ${DEPLOYMENT_NAME} - items: - - key: tls_private_key - path: key - - key: tls_certificate - path: crt ---- -apiVersion: v1 -kind: Service -metadata: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} -spec: - ports: - - port: 443 - targetPort: 443 - selector: - app: ${DEPLOYMENT_NAME} ---- -# add a label to the deployment's namespace so that we can exclude it -apiVersion: v1 -kind: Namespace -metadata: - name: ${NAMESPACE} - labels: - gmsa-webhook: disabled ---- -apiVersion: admissionregistration.k8s.io/v1beta1 -kind: ValidatingWebhookConfiguration -metadata: - name: ${DEPLOYMENT_NAME} -webhooks: -- name: k8s-gmsa-admission-webhook.sig-windows.k8s.io - clientConfig: - service: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} - path: "/validate" - caBundle: ${CA_BUNDLE} - rules: - - operations: ["CREATE", "UPDATE"] - apiGroups: [""] - apiVersions: ["*"] - resources: ["pods"] - failurePolicy: Fail - # don't run on ${NAMESPACE} - namespaceSelector: - matchExpressions: - - key: gmsa-webhook - operator: NotIn - values: [disabled] ---- -apiVersion: admissionregistration.k8s.io/v1beta1 -kind: MutatingWebhookConfiguration -metadata: - name: ${DEPLOYMENT_NAME} -webhooks: -- name: k8s-gmsa-admission-webhook.sig-windows.k8s.io - clientConfig: - service: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} - path: "/mutate" - caBundle: ${CA_BUNDLE} - rules: - - operations: ["CREATE"] - apiGroups: [""] - apiVersions: ["*"] - resources: ["pods"] - failurePolicy: Fail - # don't run on ${NAMESPACE} - namespaceSelector: - matchExpressions: - - key: gmsa-webhook - operator: NotIn - values: [disabled] -``` - -##### Configure cluster role and RBAC for service accounts to use GMSA credential specs -A service account that a pod is configured with needs to be authorized for the "use" verb on the desired GMSA credential spec resources. The following is a template using the default service account demonstrating the cluster role and role binding to use a GMSA credential spec from above. Save the file as gmsa-roles.yml and apply using `kubectl apply -f gmsa-roles.yml` - -``` -#Create the Role to read the credspec -kind: ClusterRole -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: my-rbac-reader -rules: -- apiGroups: ["windows.k8s.io"] - resources: ["gmsacredentialspecs"] - verbs: ["use"] - resourceNames: ["gmsa-WebApp1"] ---- -#Assign the role to the service account -kind: RoleBinding -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: allow-default-svc-account-read-on-gmsa-WebApp1 - namespace: default -subjects: -- kind: ServiceAccount - name: default - namespace: default -roleRef: - kind: ClusterRole - name: my-rbac-reader - apiGroup: rbac.authorization.k8s.io -``` - -##### Configure GMSA credential spec reference in pod spec -In the alpha stage of the feature, the annotation `pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name` is used to specify references to desired GMSA credential spec custom resources from pod specs. A sample pod spec with the annotation populated: - -``` -apiVersion: apps/v1beta1 -kind: Deployment -metadata: - labels: - run: with-creds - name: with-creds - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - run: with-creds - template: - metadata: - creationTimestamp: null - labels: - run: with-creds - annotations: - pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 - spec: - containers: - - image: xxxxx/test-ad-loop - imagePullPolicy: Always - name: with-creds - nodeSelector: - beta.kubernetes.io/os: windows -``` - - ## Known Limitations for Windows Server Containers with v1.9 Some of these limitations will be addressed by the community in future releases of Kubernetes: diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md new file mode 100644 index 0000000000000..776ba0f2bf0ed --- /dev/null +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -0,0 +1,273 @@ +--- +title: Configure GMSA for Windows pods and containers +content_template: templates/task +weight: 20 +--- + +{{% capture overview %}} + +{{< feature-state for_k8s_version="v1.14" state="alpha" >}} + +This page shows how to configure [Group Managed Service Accounts](https://docs.microsoft.com/en-us/windows-server/security/group-managed-service-accounts/group-managed-service-accounts-overview) (GMSA) for pods and containers that will run on Windows nodes. Group Managed Service Accounts are a specific type of Active Directory account that provides automatic password management, simplified service principal name (SPN) management, and the ability to delegate the management to other administrators across multiple servers. + +In Kubernetes, GMSA credential specs are configured at a Kubernetes cluster-wide scope as custom resources. Windows pods, as well as individual containers within a pod, can be configured to use a GMSA for domain based functions (e.g. Kerberos authentication) when interacting with other Windows services. As of v1.14, the only container runtime interface that supports GMSA for Windows workloads is Dockershim. Implementation of GMSA through CRI and other runtimes is planned for the future. + +{{< note >}} +Currently this feature is in alpha state. While the overall goals and functionality will not change, the way in which the GMSA credspec references are specified in pod specs may change from annotations to a API fields. Please take this into consideration when testing or adopting this feature. +{{< /note >}} + +{{% /capture %}} + +{{% capture body %}} + +## Setup and configuration for GMSA +Configuring GMSA credential specs in the cluster and configuring individual pods and containers to be able to use them requires several steps described in details below: +1. Enable the `WindowsGMSA` feature gate on kubelet on Windows nodes. +2. Install the GMSACredentialSpec CRD +3. Create GMSA credential spec resources +4. Install webhooks to expand and validate references to GMSA credential spec resources from pod specs +5. Create cluster roles to allow service accounts to use specific GMSA credential spec resources +6. Bind roles to specific service accounts to allow them to use GMSA credential spec resources +7. Configure pods to use GMSA credential specs along with a service account authorized to use the GMSA credential specs + +### Enable the WindowsGMSA feature gate +In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. See [Feature Gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling feature gates. Please make sure `--feature-gates=WindowsGMSA=true` parameter exists in the kubelet.exe command line. + +### Install the GMSACredentialSpec CRD +A [CustomResourceDefinition](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/) (CRD) for GMSA credential spec resources needs to be configured on the cluster to define the custom resource type `GMSACredentialSpec`. Download the GMSA CRD [YAML](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl#L131-L148) and save it as gmsa-crd.yaml. +Next, install the CRD with `kubectl apply -f gmsa-crd.yaml` + +### Create GMSA credspec resources +With the GMSACredentialSpec CRD installed, GMSA credspec custom resources can now be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Following are the steps for generating a GMSA credspec YAML based on the JSON: +1. Import the CredentialSpec module: `ipmo CredentialSpec.psm1` +2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credspec named WebApp1: `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` +3. Use `Get-CredentialSpec` to show the path of the JSON file. +4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource. An example based on a GMSA credspec named WebApp1 is below: + +``` +apiVersion: windows.k8s.io/v1alpha1 +kind: GMSACredentialSpec +metadata: + name: gmsa-WebApp1 #This is an arbitrary name but it will be used as a reference +credspec: + ActiveDirectoryConfig: + GroupManagedServiceAccounts: + - Name: WebApp1 #Username of the GMSA account + Scope: CONTOSO #NETBIOS Domain Name + - Name: WebApp1 #Username of the GMSA account + Scope: contoso.com #DNS Domain Name + CmsPlugins: + - ActiveDirectory + DomainJoinConfig: + DnsName: contoso.com #DNS Domain Name + DnsTreeName: contoso.com #DNS Domain Name Root + Guid: 244818ae-87ac-4fcd-92ec-e79e5252348a #GUID + MachineAccountName: WebApp1 #Username of the GMSA account + NetBiosName: CONTOSO #NETBIOS Domain Name + Sid: S-1-5-21-2126449477-2524075714-3094792973 #SID of GMSA +``` + +5. Deploy the credential spec resource: `kubectl apply -f gmsa-Webapp1-credspec.yml` + +[PowerShell]: https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1 + +### Install Webhooks for GMSA +Two webhooks need to be configured on the Kubernetes cluster to populate and validate GMSA credential spec references at the pod or container level. The mutating webhook expands references to GMSAs (by name from a pod specification) into the full credential spec in JSON form within the pod spec. The validating webhook ensures all references to GMSAs are authorized to be used by the pod service account. + +Installing the webhooks require several steps: +1. Create a certificate key pair (that will be used to allow the webhook container to communicate to the cluster) +2. Install a secret with the certificate from above. +3. Create a deployment for the core webhook logic. +4. Create the validating and mutating webhook configurations referring to the deployment. + +The YAML file below can be used to configure the webhooks and the associated deployment and secret. Replace the values in curly braces {} with desired values, save it as `gmsa-webhooks.yml` and apply using `kubectl apply -f gmsa-webhooks.yml` + +``` +apiVersion: v1 +kind: Secret +metadata: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} +data: + tls_private_key: ${TLS_PRIVATE_KEY} + tls_certificate: ${TLS_CERTIFICATE} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} +spec: + replicas: 1 + selector: + matchLabels: + app: ${DEPLOYMENT_NAME} + template: + metadata: + labels: + app: ${DEPLOYMENT_NAME} + spec: + containers: + - name: ${DEPLOYMENT_NAME} + image: k8ssigwindows/gmsa-admission-webhook + imagePullPolicy: IfNotPresent + ports: + - containerPort: 443 + volumeMounts: + - name: tls + mountPath: "/tls" + readOnly: true + env: + - name: TLS_KEY + value: /tls/key + - name: TLS_CRT + value: /tls/crt + volumes: + - name: tls + secret: + secretName: ${DEPLOYMENT_NAME} + items: + - key: tls_private_key + path: key + - key: tls_certificate + path: crt +--- +apiVersion: v1 +kind: Service +metadata: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} +spec: + ports: + - port: 443 + targetPort: 443 + selector: + app: ${DEPLOYMENT_NAME} +--- +# add a label to the deployment's namespace so that we can exclude it +apiVersion: v1 +kind: Namespace +metadata: + name: ${NAMESPACE} + labels: + gmsa-webhook: disabled +--- +apiVersion: admissionregistration.k8s.io/v1beta1 +kind: ValidatingWebhookConfiguration +metadata: + name: ${DEPLOYMENT_NAME} +webhooks: +- name: k8s-gmsa-admission-webhook.sig-windows.k8s.io + clientConfig: + service: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} + path: "/validate" + caBundle: ${CA_BUNDLE} + rules: + - operations: ["CREATE", "UPDATE"] + apiGroups: [""] + apiVersions: ["*"] + resources: ["pods"] + failurePolicy: Fail + # don't run on ${NAMESPACE} + namespaceSelector: + matchExpressions: + - key: gmsa-webhook + operator: NotIn + values: [disabled] +--- +apiVersion: admissionregistration.k8s.io/v1beta1 +kind: MutatingWebhookConfiguration +metadata: + name: ${DEPLOYMENT_NAME} +webhooks: +- name: k8s-gmsa-admission-webhook.sig-windows.k8s.io + clientConfig: + service: + name: ${DEPLOYMENT_NAME} + namespace: ${NAMESPACE} + path: "/mutate" + caBundle: ${CA_BUNDLE} + rules: + - operations: ["CREATE"] + apiGroups: [""] + apiVersions: ["*"] + resources: ["pods"] + failurePolicy: Fail + # don't run on ${NAMESPACE} + namespaceSelector: + matchExpressions: + - key: gmsa-webhook + operator: NotIn + values: [disabled] +``` + +### Configure cluster role to enable RBAC on specific GMSA credential specs +A cluster role needs to be defined for each GMSA that authorizes the `use` verb on a specific GMSA resource by a subject such as a service account. The following shows an example of a cluster role that authorizes usage of gmsa-WebApp1 credspec from above. Save the file as gmsa-webapp1-role.yaml and apply using `kubectl apply -f gmsa-webapp1-role.yaml` + +``` +#Create the Role to read the credspec +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: my-rbac-reader +rules: +- apiGroups: ["windows.k8s.io"] + resources: ["gmsacredentialspecs"] + verbs: ["use"] + resourceNames: ["gmsa-WebApp1"] +``` + +### Assign role to service accounts to use specific GMSA credspecs +A service account that a pod is configured with needs to be bound to the role to `use` the desired GMSA credential spec resources so that a pod's containers can be configured to use the GMSA. The following demonstrates the default service account being bound to a role to use a GMSA credential spec from above. + +``` +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: allow-default-svc-account-read-on-gmsa-WebApp1 + namespace: default +subjects: +- kind: ServiceAccount + name: default + namespace: default +roleRef: + kind: ClusterRole + name: my-rbac-reader + apiGroup: rbac.authorization.k8s.io +``` + +### Configure GMSA credential spec reference in pod spec +In the alpha stage of the feature, the annotation `pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name` is used to specify references to desired GMSA credential spec custom resources from pod specs. A sample pod spec with the annotation populated: + +``` +apiVersion: apps/v1beta1 +kind: Deployment +metadata: + labels: + run: with-creds + name: with-creds + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + run: with-creds + template: + metadata: + creationTimestamp: null + labels: + run: with-creds + annotations: + pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 + spec: + containers: + - image: xxxxx/test-ad-loop + imagePullPolicy: Always + name: with-creds + nodeSelector: + beta.kubernetes.io/os: windows +``` + +{{% /capture %}} From 70ba64d8b9238b72b8e25f04172b287d1c4c45f0 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Tue, 12 Mar 2019 19:11:31 -0700 Subject: [PATCH 07/16] Add example for container spec Signed-off-by: Deep Debroy --- .../configure-pod-container/configure-gmsa.md | 37 +++++++++++++++++-- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index 776ba0f2bf0ed..f6a225b614aee 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -239,7 +239,7 @@ roleRef: ``` ### Configure GMSA credential spec reference in pod spec -In the alpha stage of the feature, the annotation `pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name` is used to specify references to desired GMSA credential spec custom resources from pod specs. A sample pod spec with the annotation populated: +In the alpha stage of the feature, the annotation `pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name` is used to specify references to desired GMSA credential spec custom resources from pod specs. This configures all containers in the podspec to use the specified GMSA. A sample pod spec with the annotation populated: ``` apiVersion: apps/v1beta1 @@ -256,16 +256,45 @@ spec: run: with-creds template: metadata: - creationTimestamp: null labels: run: with-creds annotations: pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 spec: containers: - - image: xxxxx/test-ad-loop + - image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019 imagePullPolicy: Always - name: with-creds + name: iis + nodeSelector: + beta.kubernetes.io/os: windows +``` + +Individual containers in a pod spec can also specify the desired GMSA credspec using annotation `.container.alpha.windows.kubernetes.io/gmsa-credential-spec`. For example: + +``` +apiVersion: apps/v1beta1 +kind: Deployment +metadata: + labels: + run: with-creds + name: with-creds + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + run: with-creds + template: + metadata: + labels: + run: with-creds + annotations: + iis.container.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 + spec: + containers: + - image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019 + imagePullPolicy: Always + name: iis nodeSelector: beta.kubernetes.io/os: windows ``` From 019241a361194440194abadc14e22e182c73d2b0 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Tue, 12 Mar 2019 19:12:27 -0700 Subject: [PATCH 08/16] Remove changes in Windows index Signed-off-by: Deep Debroy --- content/en/docs/getting-started-guides/windows/_index.md | 1 - 1 file changed, 1 deletion(-) diff --git a/content/en/docs/getting-started-guides/windows/_index.md b/content/en/docs/getting-started-guides/windows/_index.md index a136755e139ca..5d0ca0d122804 100644 --- a/content/en/docs/getting-started-guides/windows/_index.md +++ b/content/en/docs/getting-started-guides/windows/_index.md @@ -297,7 +297,6 @@ This example assumes you are running on Windows Server 1709, so uses the image t ### Secrets and ConfigMaps Secrets and ConfigMaps can be utilized in Windows Server Containers, but must be used as environment variables. See limitations section below for additional details. - **Examples:** Windows pod with secrets mapped to environment variables From aec8ed84c73b2e834687c4f014fc442eda3c5451 Mon Sep 17 00:00:00 2001 From: Jeremy Wood <36464809+JeremyWx@users.noreply.github.com> Date: Fri, 15 Mar 2019 11:08:22 -0400 Subject: [PATCH 09/16] Update configure-gmsa.md --- .../configure-pod-container/configure-gmsa.md | 92 ++++++++++--------- 1 file changed, 49 insertions(+), 43 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index f6a225b614aee..661dc780310df 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -22,13 +22,17 @@ Currently this feature is in alpha state. While the overall goals and functional ## Setup and configuration for GMSA Configuring GMSA credential specs in the cluster and configuring individual pods and containers to be able to use them requires several steps described in details below: -1. Enable the `WindowsGMSA` feature gate on kubelet on Windows nodes. -2. Install the GMSACredentialSpec CRD -3. Create GMSA credential spec resources -4. Install webhooks to expand and validate references to GMSA credential spec resources from pod specs -5. Create cluster roles to allow service accounts to use specific GMSA credential spec resources -6. Bind roles to specific service accounts to allow them to use GMSA credential spec resources -7. Configure pods to use GMSA credential specs along with a service account authorized to use the GMSA credential specs + +To enable GMSA support on your cluster, there are three one-time steps to perform: +1. Enable the `WindowsGMSA` feature gate on kubelet on the Windows nodes you'll use to run GMSA-dependent workloads. +2. Install the GMSACredentialSpec Custom Resource Definition (CRD). +3. Install GMSA admission webhooks to expand and validate references to GMSA credential spec resources from pod specs + +Then, for each deployment using GMSA credentials you'll need to: +1. Create GMSA credential spec resources +2. Create cluster roles to allow service accounts to use specific GMSA credential spec resources +3. Bind roles to specific service accounts to allow them to use GMSA credential spec resources +4. Configure pods to use GMSA credential specs along with a service account authorized to use the GMSA credential specs ### Enable the WindowsGMSA feature gate In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. See [Feature Gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling feature gates. Please make sure `--feature-gates=WindowsGMSA=true` parameter exists in the kubelet.exe command line. @@ -37,40 +41,6 @@ In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubele A [CustomResourceDefinition](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/) (CRD) for GMSA credential spec resources needs to be configured on the cluster to define the custom resource type `GMSACredentialSpec`. Download the GMSA CRD [YAML](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl#L131-L148) and save it as gmsa-crd.yaml. Next, install the CRD with `kubectl apply -f gmsa-crd.yaml` -### Create GMSA credspec resources -With the GMSACredentialSpec CRD installed, GMSA credspec custom resources can now be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Following are the steps for generating a GMSA credspec YAML based on the JSON: -1. Import the CredentialSpec module: `ipmo CredentialSpec.psm1` -2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credspec named WebApp1: `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` -3. Use `Get-CredentialSpec` to show the path of the JSON file. -4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource. An example based on a GMSA credspec named WebApp1 is below: - -``` -apiVersion: windows.k8s.io/v1alpha1 -kind: GMSACredentialSpec -metadata: - name: gmsa-WebApp1 #This is an arbitrary name but it will be used as a reference -credspec: - ActiveDirectoryConfig: - GroupManagedServiceAccounts: - - Name: WebApp1 #Username of the GMSA account - Scope: CONTOSO #NETBIOS Domain Name - - Name: WebApp1 #Username of the GMSA account - Scope: contoso.com #DNS Domain Name - CmsPlugins: - - ActiveDirectory - DomainJoinConfig: - DnsName: contoso.com #DNS Domain Name - DnsTreeName: contoso.com #DNS Domain Name Root - Guid: 244818ae-87ac-4fcd-92ec-e79e5252348a #GUID - MachineAccountName: WebApp1 #Username of the GMSA account - NetBiosName: CONTOSO #NETBIOS Domain Name - Sid: S-1-5-21-2126449477-2524075714-3094792973 #SID of GMSA -``` - -5. Deploy the credential spec resource: `kubectl apply -f gmsa-Webapp1-credspec.yml` - -[PowerShell]: https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1 - ### Install Webhooks for GMSA Two webhooks need to be configured on the Kubernetes cluster to populate and validate GMSA credential spec references at the pod or container level. The mutating webhook expands references to GMSAs (by name from a pod specification) into the full credential spec in JSON form within the pod spec. The validating webhook ensures all references to GMSAs are authorized to be used by the pod service account. @@ -203,6 +173,42 @@ webhooks: values: [disabled] ``` +### Create GMSA credspec resources +With the GMSACredentialSpec CRD installed, GMSA credspec custom resources can now be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Following are the steps for generating a GMSA credspec YAML based on the JSON: +1. Import the CredentialSpec module: `ipmo CredentialSpec.psm1` +2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credspec named WebApp1: `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` +3. Use `Get-CredentialSpec` to show the path of the JSON file. +4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource. An example based on a GMSA credspec named WebApp1 is below: + +``` +apiVersion: windows.k8s.io/v1alpha1 +kind: GMSACredentialSpec +metadata: + name: gmsa-WebApp1 #This is an arbitrary name but it will be used as a reference +credspec: + ActiveDirectoryConfig: + GroupManagedServiceAccounts: + - Name: WebApp1 #Username of the GMSA account + Scope: CONTOSO #NETBIOS Domain Name + - Name: WebApp1 #Username of the GMSA account + Scope: contoso.com #DNS Domain Name + CmsPlugins: + - ActiveDirectory + DomainJoinConfig: + DnsName: contoso.com #DNS Domain Name + DnsTreeName: contoso.com #DNS Domain Name Root + Guid: 244818ae-87ac-4fcd-92ec-e79e5252348a #GUID + MachineAccountName: WebApp1 #Username of the GMSA account + NetBiosName: CONTOSO #NETBIOS Domain Name + Sid: S-1-5-21-2126449477-2524075714-3094792973 #SID of GMSA +``` + +5. Deploy the credential spec resource: `kubectl apply -f gmsa-Webapp1-credspec.yml` + +[PowerShell]: https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1 + + + ### Configure cluster role to enable RBAC on specific GMSA credential specs A cluster role needs to be defined for each GMSA that authorizes the `use` verb on a specific GMSA resource by a subject such as a service account. The following shows an example of a cluster role that authorizes usage of gmsa-WebApp1 credspec from above. Save the file as gmsa-webapp1-role.yaml and apply using `kubectl apply -f gmsa-webapp1-role.yaml` @@ -259,7 +265,7 @@ spec: labels: run: with-creds annotations: - pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 + pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 # This must be the name of the cred spec you created spec: containers: - image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019 @@ -289,7 +295,7 @@ spec: labels: run: with-creds annotations: - iis.container.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 + iis.container.alpha.windows.kubernetes.io/gmsa-credential-spec-name: gmsa-WebApp1 # This must be the name of the cred spec you created spec: containers: - image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019 From bc72a7f8aa8e802b598c2fc6b5da8f89a6f18a05 Mon Sep 17 00:00:00 2001 From: Jeremy Wood <36464809+JeremyWx@users.noreply.github.com> Date: Fri, 15 Mar 2019 15:05:51 -0400 Subject: [PATCH 10/16] Update configure-gmsa.md --- .../configure-pod-container/configure-gmsa.md | 124 +----------------- 1 file changed, 3 insertions(+), 121 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index 661dc780310df..2e4a9cad25420 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -50,128 +50,10 @@ Installing the webhooks require several steps: 3. Create a deployment for the core webhook logic. 4. Create the validating and mutating webhook configurations referring to the deployment. -The YAML file below can be used to configure the webhooks and the associated deployment and secret. Replace the values in curly braces {} with desired values, save it as `gmsa-webhooks.yml` and apply using `kubectl apply -f gmsa-webhooks.yml` +The provided script will assist you in creating the YAML file that will create the webhooks and supporting objects. [Deploy GMSA Webhook](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/deploy-gmsa-webhook.sh) + +If you wish to do this step manually or just want to see the YAML template is in entirety it can be found here: [gmsa-webhook.yml.tpl](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl) -``` -apiVersion: v1 -kind: Secret -metadata: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} -data: - tls_private_key: ${TLS_PRIVATE_KEY} - tls_certificate: ${TLS_CERTIFICATE} ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} -spec: - replicas: 1 - selector: - matchLabels: - app: ${DEPLOYMENT_NAME} - template: - metadata: - labels: - app: ${DEPLOYMENT_NAME} - spec: - containers: - - name: ${DEPLOYMENT_NAME} - image: k8ssigwindows/gmsa-admission-webhook - imagePullPolicy: IfNotPresent - ports: - - containerPort: 443 - volumeMounts: - - name: tls - mountPath: "/tls" - readOnly: true - env: - - name: TLS_KEY - value: /tls/key - - name: TLS_CRT - value: /tls/crt - volumes: - - name: tls - secret: - secretName: ${DEPLOYMENT_NAME} - items: - - key: tls_private_key - path: key - - key: tls_certificate - path: crt ---- -apiVersion: v1 -kind: Service -metadata: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} -spec: - ports: - - port: 443 - targetPort: 443 - selector: - app: ${DEPLOYMENT_NAME} ---- -# add a label to the deployment's namespace so that we can exclude it -apiVersion: v1 -kind: Namespace -metadata: - name: ${NAMESPACE} - labels: - gmsa-webhook: disabled ---- -apiVersion: admissionregistration.k8s.io/v1beta1 -kind: ValidatingWebhookConfiguration -metadata: - name: ${DEPLOYMENT_NAME} -webhooks: -- name: k8s-gmsa-admission-webhook.sig-windows.k8s.io - clientConfig: - service: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} - path: "/validate" - caBundle: ${CA_BUNDLE} - rules: - - operations: ["CREATE", "UPDATE"] - apiGroups: [""] - apiVersions: ["*"] - resources: ["pods"] - failurePolicy: Fail - # don't run on ${NAMESPACE} - namespaceSelector: - matchExpressions: - - key: gmsa-webhook - operator: NotIn - values: [disabled] ---- -apiVersion: admissionregistration.k8s.io/v1beta1 -kind: MutatingWebhookConfiguration -metadata: - name: ${DEPLOYMENT_NAME} -webhooks: -- name: k8s-gmsa-admission-webhook.sig-windows.k8s.io - clientConfig: - service: - name: ${DEPLOYMENT_NAME} - namespace: ${NAMESPACE} - path: "/mutate" - caBundle: ${CA_BUNDLE} - rules: - - operations: ["CREATE"] - apiGroups: [""] - apiVersions: ["*"] - resources: ["pods"] - failurePolicy: Fail - # don't run on ${NAMESPACE} - namespaceSelector: - matchExpressions: - - key: gmsa-webhook - operator: NotIn - values: [disabled] -``` ### Create GMSA credspec resources With the GMSACredentialSpec CRD installed, GMSA credspec custom resources can now be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Following are the steps for generating a GMSA credspec YAML based on the JSON: From 266a19322a155c1d7741bd992fa1c77a553432c8 Mon Sep 17 00:00:00 2001 From: Jeremy Wood <36464809+JeremyWx@users.noreply.github.com> Date: Fri, 15 Mar 2019 15:12:55 -0400 Subject: [PATCH 11/16] Update configure-gmsa.md --- content/en/docs/tasks/configure-pod-container/configure-gmsa.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index 2e4a9cad25420..e0382eea9bcba 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -51,6 +51,7 @@ Installing the webhooks require several steps: 4. Create the validating and mutating webhook configurations referring to the deployment. The provided script will assist you in creating the YAML file that will create the webhooks and supporting objects. [Deploy GMSA Webhook](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/deploy-gmsa-webhook.sh) +The script can be ran with the ```--dry-run``` option to allow you to review the changes that would be made to your cluster. If you wish to do this step manually or just want to see the YAML template is in entirety it can be found here: [gmsa-webhook.yml.tpl](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl) From ada9ae585b00cfc3fd3c08d8b80872bc505f8050 Mon Sep 17 00:00:00 2001 From: Jeremy Wood <36464809+JeremyWx@users.noreply.github.com> Date: Fri, 15 Mar 2019 15:13:44 -0400 Subject: [PATCH 12/16] Update configure-gmsa.md --- content/en/docs/tasks/configure-pod-container/configure-gmsa.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index e0382eea9bcba..8e09b3dc226c9 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -50,7 +50,7 @@ Installing the webhooks require several steps: 3. Create a deployment for the core webhook logic. 4. Create the validating and mutating webhook configurations referring to the deployment. -The provided script will assist you in creating the YAML file that will create the webhooks and supporting objects. [Deploy GMSA Webhook](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/deploy-gmsa-webhook.sh) +The provided script will assist you in creating the YAML file that will create the webhooks and supporting objects. [Deploy GMSA Webhook](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/deploy-gmsa-webhook.sh). The script can be ran with the ```--dry-run``` option to allow you to review the changes that would be made to your cluster. If you wish to do this step manually or just want to see the YAML template is in entirety it can be found here: [gmsa-webhook.yml.tpl](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl) From 29f321dbf0ac693ee20c2529beaa52ec7980a849 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Fri, 15 Mar 2019 13:42:02 -0700 Subject: [PATCH 13/16] Rearrange the steps into two sections and other edits Signed-off-by: Deep Debroy --- .../configure-pod-container/configure-gmsa.md | 59 +++++++++---------- 1 file changed, 28 insertions(+), 31 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index 8e09b3dc226c9..f2f10bb100fe6 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -21,47 +21,47 @@ Currently this feature is in alpha state. While the overall goals and functional {{% capture body %}} ## Setup and configuration for GMSA -Configuring GMSA credential specs in the cluster and configuring individual pods and containers to be able to use them requires several steps described in details below: +Configuring GMSA credential specs in the cluster and configuring individual pods and containers to be able to use them requires several steps described in details below. -To enable GMSA support on your cluster, there are three one-time steps to perform: -1. Enable the `WindowsGMSA` feature gate on kubelet on the Windows nodes you'll use to run GMSA-dependent workloads. -2. Install the GMSACredentialSpec Custom Resource Definition (CRD). -3. Install GMSA admission webhooks to expand and validate references to GMSA credential spec resources from pod specs +### Initial configuration of Kubernetes cluster to use GMSA +This section covers a set of initial steps required once for each cluster. These include: +1. Enabling the `WindowsGMSA` feature gate on kubelet on the Windows nodes you'll use to run GMSA-dependent workloads. +2. Installing the GMSACredentialSpec Custom Resource Definition (CRD). +3. Installing two GMSA admission webhooks to expand and validate references to GMSA credential spec resources from pod specs. -Then, for each deployment using GMSA credentials you'll need to: -1. Create GMSA credential spec resources -2. Create cluster roles to allow service accounts to use specific GMSA credential spec resources -3. Bind roles to specific service accounts to allow them to use GMSA credential spec resources -4. Configure pods to use GMSA credential specs along with a service account authorized to use the GMSA credential specs - -### Enable the WindowsGMSA feature gate +#### Enable the WindowsGMSA feature gate In the alpha state, the `WindowsGMSA` feature gate needs to be enabled on kubelet on Windows nodes. This is required to pass down the GMSA credential specs from the cluster scoped configurations to the container runtime. See [Feature Gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling feature gates. Please make sure `--feature-gates=WindowsGMSA=true` parameter exists in the kubelet.exe command line. -### Install the GMSACredentialSpec CRD +#### Install the GMSACredentialSpec CRD A [CustomResourceDefinition](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/) (CRD) for GMSA credential spec resources needs to be configured on the cluster to define the custom resource type `GMSACredentialSpec`. Download the GMSA CRD [YAML](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl#L131-L148) and save it as gmsa-crd.yaml. Next, install the CRD with `kubectl apply -f gmsa-crd.yaml` -### Install Webhooks for GMSA +#### Install webhooks to validate GMSA users Two webhooks need to be configured on the Kubernetes cluster to populate and validate GMSA credential spec references at the pod or container level. The mutating webhook expands references to GMSAs (by name from a pod specification) into the full credential spec in JSON form within the pod spec. The validating webhook ensures all references to GMSAs are authorized to be used by the pod service account. -Installing the webhooks require several steps: +Installing the webhooks and associated objects require the steps below: 1. Create a certificate key pair (that will be used to allow the webhook container to communicate to the cluster) 2. Install a secret with the certificate from above. 3. Create a deployment for the core webhook logic. 4. Create the validating and mutating webhook configurations referring to the deployment. -The provided script will assist you in creating the YAML file that will create the webhooks and supporting objects. [Deploy GMSA Webhook](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/deploy-gmsa-webhook.sh). -The script can be ran with the ```--dry-run``` option to allow you to review the changes that would be made to your cluster. +A [script] (https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/deploy-gmsa-webhook.sh) can be used to deploy and configure the GMSA webhooks and associated objects mentioned above. The script can be run with a ```--dry-run``` option to allow you to review the changes that would be made to your cluster. -If you wish to do this step manually or just want to see the YAML template is in entirety it can be found here: [gmsa-webhook.yml.tpl](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl) +The [YAML template] (https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl) used by the script may also be used to deploy the webhooks and associated objects manually (with appropriate substitutions for the parameters) +### Configuration and usage of GMSAs in pods +This section covers the set of steps necessary for configuring individual GMSA credentials and using them in pods. These include: +1. Creating GMSA credential spec resources. +2. Creating cluster roles to allow service accounts to use specific GMSA credential spec resources. +3. Binding roles to specific service accounts to allow them to use the desired GMSA credential spec resources. +4. Configuring pods with a service account authorized to use the desired GMSA credential specs. -### Create GMSA credspec resources -With the GMSACredentialSpec CRD installed, GMSA credspec custom resources can now be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. The GMSA credspec resources can be generated in JSON format with a utility [PowerShell][] script. Following are the steps for generating a GMSA credspec YAML based on the JSON: -1. Import the CredentialSpec module: `ipmo CredentialSpec.psm1` -2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credspec named WebApp1: `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` +#### Create GMSA credspec resources +With the GMSACredentialSpec CRD installed, custom resources containing GMSA credential specs can be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. GMSA credential specs can be generated in JSON format with a utility [PowerShell script](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1). Following are the steps for generating a GMSA credential spec YAML based on the JSON: +1. Import the CredentialSpec [module](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1): `ipmo CredentialSpec.psm1` +2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credential spec named WebApp1, invoke `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` 3. Use `Get-CredentialSpec` to show the path of the JSON file. -4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource. An example based on a GMSA credspec named WebApp1 is below: +4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource that can be configured in Kubernetes. An example based on a GMSA credential spec named WebApp1 is below: ``` apiVersion: windows.k8s.io/v1alpha1 @@ -88,12 +88,9 @@ credspec: 5. Deploy the credential spec resource: `kubectl apply -f gmsa-Webapp1-credspec.yml` -[PowerShell]: https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1 - - -### Configure cluster role to enable RBAC on specific GMSA credential specs -A cluster role needs to be defined for each GMSA that authorizes the `use` verb on a specific GMSA resource by a subject such as a service account. The following shows an example of a cluster role that authorizes usage of gmsa-WebApp1 credspec from above. Save the file as gmsa-webapp1-role.yaml and apply using `kubectl apply -f gmsa-webapp1-role.yaml` +#### Configure cluster role to enable RBAC on specific GMSA credential specs +A cluster role needs to be defined for each GMSA that authorizes the `use` verb on a specific GMSA resource by a subject such as a service account. The following shows an example of a cluster role that authorizes usage of gmsa-WebApp1 credential spec from above. Save the file as gmsa-webapp1-role.yaml and apply using `kubectl apply -f gmsa-webapp1-role.yaml` ``` #Create the Role to read the credspec @@ -108,8 +105,8 @@ rules: resourceNames: ["gmsa-WebApp1"] ``` -### Assign role to service accounts to use specific GMSA credspecs -A service account that a pod is configured with needs to be bound to the role to `use` the desired GMSA credential spec resources so that a pod's containers can be configured to use the GMSA. The following demonstrates the default service account being bound to a role to use a GMSA credential spec from above. +#### Assign role to service accounts to use specific GMSA credspecs +A service account that a pod is configured with needs to be bound to the role create above. This authorizes the service account to "use" the desired GMSA credential spec resource. The following demonstrates the default service account being bound to a role to use a GMSA credential spec from above. ``` kind: RoleBinding @@ -127,7 +124,7 @@ roleRef: apiGroup: rbac.authorization.k8s.io ``` -### Configure GMSA credential spec reference in pod spec +#### Configure GMSA credential spec reference in pod spec In the alpha stage of the feature, the annotation `pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name` is used to specify references to desired GMSA credential spec custom resources from pod specs. This configures all containers in the podspec to use the specified GMSA. A sample pod spec with the annotation populated: ``` From 84734b0913ac72543961f1d24e57d0718476467f Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Fri, 15 Mar 2019 13:46:25 -0700 Subject: [PATCH 14/16] Fix links Signed-off-by: Deep Debroy --- .../tasks/configure-pod-container/configure-gmsa.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index f2f10bb100fe6..b96a0e5d73d74 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -37,17 +37,19 @@ A [CustomResourceDefinition](https://kubernetes.io/docs/tasks/access-kubernetes- Next, install the CRD with `kubectl apply -f gmsa-crd.yaml` #### Install webhooks to validate GMSA users -Two webhooks need to be configured on the Kubernetes cluster to populate and validate GMSA credential spec references at the pod or container level. The mutating webhook expands references to GMSAs (by name from a pod specification) into the full credential spec in JSON form within the pod spec. The validating webhook ensures all references to GMSAs are authorized to be used by the pod service account. +Two webhooks need to be configured on the Kubernetes cluster to populate and validate GMSA credential spec references at the pod or container level: +1. A mutating webhook that expands references to GMSAs (by name from a pod specification) into the full credential spec in JSON form within the pod spec. +2. A validating webhook ensures all references to GMSAs are authorized to be used by the pod service account. -Installing the webhooks and associated objects require the steps below: +Installing the above webhooks and associated objects require the steps below: 1. Create a certificate key pair (that will be used to allow the webhook container to communicate to the cluster) 2. Install a secret with the certificate from above. 3. Create a deployment for the core webhook logic. 4. Create the validating and mutating webhook configurations referring to the deployment. -A [script] (https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/deploy-gmsa-webhook.sh) can be used to deploy and configure the GMSA webhooks and associated objects mentioned above. The script can be run with a ```--dry-run``` option to allow you to review the changes that would be made to your cluster. +A [script](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/deploy-gmsa-webhook.sh) can be used to deploy and configure the GMSA webhooks and associated objects mentioned above. The script can be run with a ```--dry-run``` option to allow you to review the changes that would be made to your cluster. -The [YAML template] (https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl) used by the script may also be used to deploy the webhooks and associated objects manually (with appropriate substitutions for the parameters) +The [YAML template](https://github.com/kubernetes-sigs/windows-gmsa/blob/master/admission-webhook/deploy/gmsa-webhook.yml.tpl) used by the script may also be used to deploy the webhooks and associated objects manually (with appropriate substitutions for the parameters) ### Configuration and usage of GMSAs in pods This section covers the set of steps necessary for configuring individual GMSA credentials and using them in pods. These include: From 35ca428629526bb09dd86de5c03c27a300556775 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Fri, 15 Mar 2019 14:02:09 -0700 Subject: [PATCH 15/16] Add reference to script to generate GMSA YAMLs Signed-off-by: Deep Debroy --- .../en/docs/tasks/configure-pod-container/configure-gmsa.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index b96a0e5d73d74..dd60eebfa6aac 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -59,7 +59,9 @@ This section covers the set of steps necessary for configuring individual GMSA c 4. Configuring pods with a service account authorized to use the desired GMSA credential specs. #### Create GMSA credspec resources -With the GMSACredentialSpec CRD installed, custom resources containing GMSA credential specs can be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. GMSA credential specs can be generated in JSON format with a utility [PowerShell script](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1). Following are the steps for generating a GMSA credential spec YAML based on the JSON: +With the GMSACredentialSpec CRD installed, custom resources containing GMSA credential specs can be configured. The GMSA credential spec does not contain secret or sensitive data. It is information that a container runtime can use to describe the desired GMSA of a container to Windows. GMSA credential specs can be generated in YAML format with a utility [PowerShell script](https://github.com/kubernetes-sigs/windows-gmsa/tree/master/scripts/GenerateCredentialSpecResource.ps1). + +Following are the steps for generating a GMSA credential spec YAML manually in JSON format and then converting it: 1. Import the CredentialSpec [module](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1): `ipmo CredentialSpec.psm1` 2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credential spec named WebApp1, invoke `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` 3. Use `Get-CredentialSpec` to show the path of the JSON file. From 59b50741044090a4a4deceb0ab7a49b8fa1c86d7 Mon Sep 17 00:00:00 2001 From: Deep Debroy Date: Fri, 15 Mar 2019 23:26:43 -0700 Subject: [PATCH 16/16] Some more clarifications for GMSA Signed-off-by: Deep Debroy --- .../tasks/configure-pod-container/configure-gmsa.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index dd60eebfa6aac..844db949c405e 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -65,7 +65,9 @@ Following are the steps for generating a GMSA credential spec YAML manually in J 1. Import the CredentialSpec [module](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/windows-server-container-tools/ServiceAccounts/CredentialSpec.psm1): `ipmo CredentialSpec.psm1` 2. Create a credential spec in JSON format using `New-CredentialSpec`. To create a GMSA credential spec named WebApp1, invoke `New-CredentialSpec -Name WebApp1 -AccountName WebApp1 -Domain $(Get-ADDomain -Current LocalComputer)` 3. Use `Get-CredentialSpec` to show the path of the JSON file. -4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource that can be configured in Kubernetes. An example based on a GMSA credential spec named WebApp1 is below: +4. Convert the credspec file from JSON to YAML format and apply the necessary header fields `apiVersion`, `kind`, `metadata` and `credspec` to make it a GMSACredentialSpec custom resource that can be configured in Kubernetes. + +The following YAML configuration describes a GMSA credential spec named `gmsa-WebApp1`: ``` apiVersion: windows.k8s.io/v1alpha1 @@ -94,14 +96,14 @@ credspec: #### Configure cluster role to enable RBAC on specific GMSA credential specs -A cluster role needs to be defined for each GMSA that authorizes the `use` verb on a specific GMSA resource by a subject such as a service account. The following shows an example of a cluster role that authorizes usage of gmsa-WebApp1 credential spec from above. Save the file as gmsa-webapp1-role.yaml and apply using `kubectl apply -f gmsa-webapp1-role.yaml` +A cluster role needs to be defined for each GMSA credential spec resource. This authorizes the `use` verb on a specific GMSA resource by a subject which is typically a service account. The following example shows a cluster role that authorizes usage of the `gmsa-WebApp1` credential spec from above. Save the file as gmsa-webapp1-role.yaml and apply using `kubectl apply -f gmsa-webapp1-role.yaml` ``` #Create the Role to read the credspec kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: - name: my-rbac-reader + name: webapp1-role rules: - apiGroups: ["windows.k8s.io"] resources: ["gmsacredentialspecs"] @@ -110,7 +112,7 @@ rules: ``` #### Assign role to service accounts to use specific GMSA credspecs -A service account that a pod is configured with needs to be bound to the role create above. This authorizes the service account to "use" the desired GMSA credential spec resource. The following demonstrates the default service account being bound to a role to use a GMSA credential spec from above. +A service account (that pods will be configured with) needs to be bound to the cluster role create above. This authorizes the service account to "use" the desired GMSA credential spec resource. The following shows the default service account being bound to a cluster role `webapp1-role` to use `gmsa-WebApp1` credential spec resource created above. ``` kind: RoleBinding @@ -129,7 +131,7 @@ roleRef: ``` #### Configure GMSA credential spec reference in pod spec -In the alpha stage of the feature, the annotation `pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name` is used to specify references to desired GMSA credential spec custom resources from pod specs. This configures all containers in the podspec to use the specified GMSA. A sample pod spec with the annotation populated: +In the alpha stage of the feature, the annotation `pod.alpha.windows.kubernetes.io/gmsa-credential-spec-name` is used to specify references to desired GMSA credential spec custom resources from pod specs. This configures all containers in the podspec to use the specified GMSA. A sample pod spec with the annotation populated to refer to `gmsa-WebApp1`: ``` apiVersion: apps/v1beta1