From b7343ac6eb6ff12a44e888026c84dd7e26d85254 Mon Sep 17 00:00:00 2001
From: Tim Allclair <tallclair@google.com>
Date: Fri, 8 Mar 2019 11:18:31 -0800
Subject: [PATCH 1/5] Update RuntimeClass documentation for beta

---
 .../docs/concepts/containers/runtime-class.md | 109 +++++++++++-------
 1 file changed, 66 insertions(+), 43 deletions(-)

diff --git a/content/en/docs/concepts/containers/runtime-class.md b/content/en/docs/concepts/containers/runtime-class.md
index 861ce41d9e4b6..ff3772f130cbe 100644
--- a/content/en/docs/concepts/containers/runtime-class.md
+++ b/content/en/docs/concepts/containers/runtime-class.md
@@ -9,7 +9,7 @@ weight: 20
 
 {{% capture overview %}}
 
-{{< feature-state for_k8s_version="v1.12" state="alpha" >}}
+{{< feature-state for_k8s_version="v1.14" state="beta" >}}
 
 This page describes the RuntimeClass resource and runtime selection mechanism.
 
@@ -20,72 +20,61 @@ This page describes the RuntimeClass resource and runtime selection mechanism.
 
 ## Runtime Class
 
-RuntimeClass is an alpha feature for selecting the container runtime configuration to use to run a
-pod's containers.
+RuntimeClass is a beta feature (as of v1.14) for selecting the container runtime configuration to
+use to run a pod's containers.
 
-### Set Up
-
-As an early alpha feature, there are some additional setup steps that must be taken in order to use
-the RuntimeClass feature:
-
-1. Enable the RuntimeClass feature gate (on apiservers & kubelets, requires version 1.12+)
-2. Install the RuntimeClass CRD
-3. Configure the CRI implementation on nodes (runtime dependent)
-4. Create the corresponding RuntimeClass resources
+{{< warning >}} RuntimeClass includes *breaking* changes in the beta upgrade in v1.14. If you were
+using RuntimeClass prior to v1.14, the following actions are required:
 
-#### 1. Enable the RuntimeClass feature gate
+- The `node.k8s.io` API group and `runtimeclasses.node.k8s.io` resource have been migrated to a
+  built-in API. RuntimeClass resources must be recreated *after* upgrading to v1.14, and the
+  runtimeclasses.node.k8s.io CRD should be manually deleted.
+- The `spec.runtimeHandler` field in v1aplha1 has been renamed to `handler` (no more `spec`). The
+  handler is now a required field. Handlers can no longer include `.` characters.
 
-See [Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation
-of enabling feature gates. The `RuntimeClass` feature gate must be enabled on apiservers _and_
-kubelets.
+{{< /warning >}}
 
-#### 2. Install the RuntimeClass CRD
-
-The RuntimeClass [CustomResourceDefinition][] (CRD) can be found in the addons directory of the
-Kubernetes git repo: [kubernetes/cluster/addons/runtimeclass/runtimeclass_crd.yaml][runtimeclass_crd]
-
-Install the CRD with `kubectl apply -f runtimeclass_crd.yaml`.
+### Set Up
 
-[CustomResourceDefinition]: /docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/
-[runtimeclass_crd]: https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/runtimeclass/runtimeclass_crd.yaml
+Prerequisite: Ensure the RuntimeClass feature gate is enabled (it is by default). See [Feature
+Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling
+feature gates. The `RuntimeClass` feature gate must be enabled on apiservers _and_ kubelets.
 
+1. Configure the CRI implementation on nodes (runtime dependent)
+2. Create the corresponding RuntimeClass resources
 
-#### 3. Configure the CRI implementation on nodes
+#### 1. Configure the CRI implementation on nodes
 
-The configurations to select between with RuntimeClass are CRI implementation dependent. See the
-corresponding documentation for your CRI implementation for how to configure. As this is an alpha
-feature, not all CRIs support multiple RuntimeClasses yet.
+The configurations available through RuntimeClass are CRI implementation dependent. See the
+corresponding documentation ([below](#cri-documentaiton)) for your CRI implementation for how to configure.
 
 {{< note >}}
-RuntimeClass currently assumes a homogeneous node configuration across the cluster
-(which means that all nodes are configured the same way with respect to container runtimes). Any heterogeneity (varying configurations) must be
-managed independently of RuntimeClass through scheduling features (see [Assigning Pods to
-Nodes](/docs/concepts/configuration/assign-pod-node/)).
+RuntimeClass currently assumes a homogeneous node configuration across the cluster (which means that
+all nodes are configured the same way with respect to container runtimes). Any heterogeneity
+(varying configurations) must be managed independently of RuntimeClass through scheduling features
+(see [Assigning Pods to Nodes](/docs/concepts/configuration/assign-pod-node/)).
 {{< /note >}}
 
-The configurations have a corresponding `RuntimeHandler` name, referenced by the RuntimeClass. The
-RuntimeHandler must be a valid DNS 1123 subdomain (alpha-numeric + `-` and `.` characters).
+The configurations have a corresponding `handler` name, referenced by the RuntimeClass. The
+handler must be a valid DNS 1123 label (alpha-numeric + `-` characters).
 
-#### 4. Create the corresponding RuntimeClass resources
+#### 2. Create the corresponding RuntimeClass resources
 
-The configurations setup in step 3 should each have an associated `RuntimeHandler` name, which
-identifies the configuration. For each RuntimeHandler (and optionally the empty `""` handler),
-create a corresponding RuntimeClass object.
+The configurations setup in step 1 should each have an associated `handler` name, which identifies
+the configuration. For each handler, create a corresponding RuntimeClass object.
 
 The RuntimeClass resource currently only has 2 significant fields: the RuntimeClass name
-(`metadata.name`) and the RuntimeHandler (`spec.runtimeHandler`). The object definition looks like this:
+(`metadata.name`) and the handler (`handler`). The object definition looks like this:
 
 ```yaml
-apiVersion: node.k8s.io/v1alpha1  # RuntimeClass is defined in the node.k8s.io API group
+apiVersion: node.k8s.io/v1beta1  # RuntimeClass is defined in the node.k8s.io API group
 kind: RuntimeClass
 metadata:
   name: myclass  # The name the RuntimeClass will be referenced by
   # RuntimeClass is a non-namespaced resource
-spec:
-  runtimeHandler: myconfiguration  # The name of the corresponding CRI configuration
+handler: myconfiguration  # The name of the corresponding CRI configuration
 ```
 
-
 {{< note >}}
 It is recommended that RuntimeClass write operations (create/update/patch/delete) be
 restricted to the cluster administrator. This is typically the default. See [Authorization
@@ -116,4 +105,38 @@ error message.
 If no `runtimeClassName` is specified, the default RuntimeHandler will be used, which is equivalent
 to the behavior when the RuntimeClass feature is disabled.
 
+### CRI Configuration
+
+For more details on setting up CRI runtimes, see [CRI installation](/docs/setup/cri/).
+
+#### dockershim
+
+Kubernete's built-in dockershim CRI does not support runtime handlers.
+
+#### [containerd](https://containerd.io/)
+
+Runtime handlers are configured through containerd's configuration at
+`/etc/containerd/config.toml`. Valid handlers are configured under the runtimes section:
+
+```
+[plugins.cri.containerd.runtimes.${HANDLER_NAME}]
+```
+
+See containerd's config documentation for more details:
+https://github.com/containerd/cri/blob/master/docs/config.md
+
+#### [cri-o](https://cri-o.io/)
+
+Runtime handlers are configured through cri-o's configuration at `/etc/crio/crio.conf`. Valid
+handlers are configured under the [crio.runtime
+table](https://github.com/kubernetes-sigs/cri-o/blob/master/docs/crio.conf.5.md#crioruntime-table):
+
+```
+[crio.runtime.runtimes.${HANDLER_NAME}]
+  runtime_path = "${PATH_TO_BINARY}"
+```
+
+See cri-o's config documentation for more details:
+https://github.com/kubernetes-sigs/cri-o/blob/master/cmd/crio/config.go
+
 {{% /capture %}}

From fe01694942e51594cbdc643933925990ddd28fc7 Mon Sep 17 00:00:00 2001
From: Tim Allclair <tallclair@google.com>
Date: Mon, 11 Mar 2019 16:17:53 -0700
Subject: [PATCH 2/5] Update feature gate & add upgrade section

---
 .../docs/concepts/containers/runtime-class.md | 40 ++++++++++++++-----
 .../feature-gates.md                          |  2 +-
 2 files changed, 32 insertions(+), 10 deletions(-)

diff --git a/content/en/docs/concepts/containers/runtime-class.md b/content/en/docs/concepts/containers/runtime-class.md
index ff3772f130cbe..c8c687fcf6fba 100644
--- a/content/en/docs/concepts/containers/runtime-class.md
+++ b/content/en/docs/concepts/containers/runtime-class.md
@@ -23,15 +23,10 @@ This page describes the RuntimeClass resource and runtime selection mechanism.
 RuntimeClass is a beta feature (as of v1.14) for selecting the container runtime configuration to
 use to run a pod's containers.
 
-{{< warning >}} RuntimeClass includes *breaking* changes in the beta upgrade in v1.14. If you were
-using RuntimeClass prior to v1.14, the following actions are required:
-
-- The `node.k8s.io` API group and `runtimeclasses.node.k8s.io` resource have been migrated to a
-  built-in API. RuntimeClass resources must be recreated *after* upgrading to v1.14, and the
-  runtimeclasses.node.k8s.io CRD should be manually deleted.
-- The `spec.runtimeHandler` field in v1aplha1 has been renamed to `handler` (no more `spec`). The
-  handler is now a required field. Handlers can no longer include `.` characters.
-
+{{< warning >}}
+RuntimeClass includes *breaking* changes in the beta upgrade in v1.14. If you were using
+RuntimeClass prior to v1.14, see [Upgrading RuntimeClass from Alpha to
+Beta](#upgrading-runtimeclass-from-alpha-to-beta).
 {{< /warning >}}
 
 ### Set Up
@@ -139,4 +134,31 @@ table](https://github.com/kubernetes-sigs/cri-o/blob/master/docs/crio.conf.5.md#
 See cri-o's config documentation for more details:
 https://github.com/kubernetes-sigs/cri-o/blob/master/cmd/crio/config.go
 
+
+### Upgrading RuntimeClass from Alpha to Beta
+
+RuntimeClass Beta (v1.14) included the following changes:
+
+- The `node.k8s.io` API group and `runtimeclasses.node.k8s.io` resource have been migrated to a
+  built-in API from a CustomResourceDefintion.
+- The `spec` has been inlined in the RuntimeClass definition (i.e. there is no more
+  RuntimeClassSpec).
+- The `runtimeHandler` field has been renamed `handler`.
+- The `handler` field is now required in all API versions. This means the `runtimeHandler` field in
+  the Alpha API is also required.
+- The `handler` field must be a valid DNS label ([RFC 1123](https://tools.ietf.org/html/rfc1123)),
+  meaning it can no longer container `.` characters (in all versions). Valid handlers match the
+  following regular expression: `^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`.
+
+If you were using RuntimeClass Alpha (prior to v1.14), the following actions are required
+
+- RuntimeClass resources must be recreated *after* upgrading to v1.14, and the
+  `runtimeclasses.node.k8s.io` CRD should be manually deleted:
+
+      kubectl delete customresourcedefinitions.apiextensions.k8s.io runtimeclasses.node.k8s.io
+
+- Alpha RuntimeClasses with an unspecified or empty `runtimeHandler` or those using a `.` character
+  in the handler are no longer valid, and must be migrated to a valid handler configuration (see
+  above).
+
 {{% /capture %}}
diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md
index 43dc0c9be627c..fac66d2ec837e 100644
--- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md
+++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md
@@ -119,7 +119,7 @@ different Kubernetes components.
 | `RotateKubeletServerCertificate` | `false` | Alpha | 1.7 | 1.11 |
 | `RotateKubeletServerCertificate` | `true` | Beta | 1.12 | |
 | `RunAsGroup` | `false` | Alpha | 1.10 | |
-| `RuntimeClass` | `false` | Alpha | 1.12 | |
+| `RuntimeClass` | `true` | Beta | 1.14 | |
 | `SCTPSupport` | `false` | Alpha | 1.12 | |
 | `ServiceNodeExclusion` | `false` | Alpha | 1.8 | |
 | `StorageObjectInUseProtection` | `true` | Beta | 1.10 | 1.10 |

From cd9e6afbac9cb7bca4269fc447ccb4079a987bb2 Mon Sep 17 00:00:00 2001
From: Tim Allclair <tallclair@google.com>
Date: Tue, 12 Mar 2019 10:23:59 -0700
Subject: [PATCH 3/5] formatting fixes

---
 .../docs/concepts/containers/runtime-class.md | 22 +++++++++----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/content/en/docs/concepts/containers/runtime-class.md b/content/en/docs/concepts/containers/runtime-class.md
index c8c687fcf6fba..1b0f7830a6207 100644
--- a/content/en/docs/concepts/containers/runtime-class.md
+++ b/content/en/docs/concepts/containers/runtime-class.md
@@ -13,6 +13,12 @@ weight: 20
 
 This page describes the RuntimeClass resource and runtime selection mechanism.
 
+{{< warning >}}
+RuntimeClass includes *breaking* changes in the beta upgrade in v1.14. If you were using
+RuntimeClass prior to v1.14, see [Upgrading RuntimeClass from Alpha to
+Beta](#upgrading-runtimeclass-from-alpha-to-beta).
+{{< /warning >}}
+
 {{% /capture %}}
 
 
@@ -23,12 +29,6 @@ This page describes the RuntimeClass resource and runtime selection mechanism.
 RuntimeClass is a beta feature (as of v1.14) for selecting the container runtime configuration to
 use to run a pod's containers.
 
-{{< warning >}}
-RuntimeClass includes *breaking* changes in the beta upgrade in v1.14. If you were using
-RuntimeClass prior to v1.14, see [Upgrading RuntimeClass from Alpha to
-Beta](#upgrading-runtimeclass-from-alpha-to-beta).
-{{< /warning >}}
-
 ### Set Up
 
 Prerequisite: Ensure the RuntimeClass feature gate is enabled (it is by default). See [Feature
@@ -106,7 +106,7 @@ For more details on setting up CRI runtimes, see [CRI installation](/docs/setup/
 
 #### dockershim
 
-Kubernete's built-in dockershim CRI does not support runtime handlers.
+Kubernetes built-in dockershim CRI does not support runtime handlers.
 
 #### [containerd](https://containerd.io/)
 
@@ -150,13 +150,13 @@ RuntimeClass Beta (v1.14) included the following changes:
   meaning it can no longer container `.` characters (in all versions). Valid handlers match the
   following regular expression: `^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`.
 
-If you were using RuntimeClass Alpha (prior to v1.14), the following actions are required
+If you were using RuntimeClass Alpha (prior to v1.14), the following **actions are required**:
 
 - RuntimeClass resources must be recreated *after* upgrading to v1.14, and the
   `runtimeclasses.node.k8s.io` CRD should be manually deleted:
-
-      kubectl delete customresourcedefinitions.apiextensions.k8s.io runtimeclasses.node.k8s.io
-
+  ```
+  kubectl delete customresourcedefinitions.apiextensions.k8s.io runtimeclasses.node.k8s.io
+  ```
 - Alpha RuntimeClasses with an unspecified or empty `runtimeHandler` or those using a `.` character
   in the handler are no longer valid, and must be migrated to a valid handler configuration (see
   above).

From caf1120c75fec2cb898634ffcd75885ad325f2e9 Mon Sep 17 00:00:00 2001
From: Tim Allclair <tallclair@google.com>
Date: Tue, 12 Mar 2019 10:33:00 -0700
Subject: [PATCH 4/5] Highlight upgrade action required

---
 content/en/docs/concepts/containers/runtime-class.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/content/en/docs/concepts/containers/runtime-class.md b/content/en/docs/concepts/containers/runtime-class.md
index 1b0f7830a6207..9f9ce4be545f6 100644
--- a/content/en/docs/concepts/containers/runtime-class.md
+++ b/content/en/docs/concepts/containers/runtime-class.md
@@ -150,7 +150,7 @@ RuntimeClass Beta (v1.14) included the following changes:
   meaning it can no longer container `.` characters (in all versions). Valid handlers match the
   following regular expression: `^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`.
 
-If you were using RuntimeClass Alpha (prior to v1.14), the following **actions are required**:
+**Action Required:** If you were using RuntimeClass Alpha (prior to v1.14), the following actions are required upon upgrade to v1.14:
 
 - RuntimeClass resources must be recreated *after* upgrading to v1.14, and the
   `runtimeclasses.node.k8s.io` CRD should be manually deleted:

From 3094961030f1ca8fe28dcf15650318a89ae0bbdd Mon Sep 17 00:00:00 2001
From: Tim Allclair <tallclair@google.com>
Date: Wed, 13 Mar 2019 16:27:14 -0700
Subject: [PATCH 5/5] Address feedback

---
 .../docs/concepts/containers/runtime-class.md | 20 ++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/content/en/docs/concepts/containers/runtime-class.md b/content/en/docs/concepts/containers/runtime-class.md
index 9f9ce4be545f6..c2ca8a830f537 100644
--- a/content/en/docs/concepts/containers/runtime-class.md
+++ b/content/en/docs/concepts/containers/runtime-class.md
@@ -26,12 +26,12 @@ Beta](#upgrading-runtimeclass-from-alpha-to-beta).
 
 ## Runtime Class
 
-RuntimeClass is a beta feature (as of v1.14) for selecting the container runtime configuration to
-use to run a pod's containers.
+RuntimeClass is a feature for selecting the container runtime configuration. The container runtime
+configuration is used to run a Pod's containers.
 
 ### Set Up
 
-Prerequisite: Ensure the RuntimeClass feature gate is enabled (it is by default). See [Feature
+Ensure the RuntimeClass feature gate is enabled (it is by default). See [Feature
 Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling
 feature gates. The `RuntimeClass` feature gate must be enabled on apiservers _and_ kubelets.
 
@@ -40,8 +40,9 @@ feature gates. The `RuntimeClass` feature gate must be enabled on apiservers _an
 
 #### 1. Configure the CRI implementation on nodes
 
-The configurations available through RuntimeClass are CRI implementation dependent. See the
-corresponding documentation ([below](#cri-documentaiton)) for your CRI implementation for how to configure.
+The configurations available through RuntimeClass are Container Runtime Interface (CRI)
+implementation dependent. See the corresponding documentation ([below](#cri-documentation)) for your
+CRI implementation for how to configure.
 
 {{< note >}}
 RuntimeClass currently assumes a homogeneous node configuration across the cluster (which means that
@@ -137,20 +138,21 @@ https://github.com/kubernetes-sigs/cri-o/blob/master/cmd/crio/config.go
 
 ### Upgrading RuntimeClass from Alpha to Beta
 
-RuntimeClass Beta (v1.14) included the following changes:
+The RuntimeClass Beta feature includes the following changes:
 
 - The `node.k8s.io` API group and `runtimeclasses.node.k8s.io` resource have been migrated to a
-  built-in API from a CustomResourceDefintion.
+  built-in API from a CustomResourceDefinition.
 - The `spec` has been inlined in the RuntimeClass definition (i.e. there is no more
   RuntimeClassSpec).
 - The `runtimeHandler` field has been renamed `handler`.
 - The `handler` field is now required in all API versions. This means the `runtimeHandler` field in
   the Alpha API is also required.
 - The `handler` field must be a valid DNS label ([RFC 1123](https://tools.ietf.org/html/rfc1123)),
-  meaning it can no longer container `.` characters (in all versions). Valid handlers match the
+  meaning it can no longer contain `.` characters (in all versions). Valid handlers match the
   following regular expression: `^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`.
 
-**Action Required:** If you were using RuntimeClass Alpha (prior to v1.14), the following actions are required upon upgrade to v1.14:
+**Action Required:** The following actions are required to upgrade from the alpha version of the
+RuntimeClass feature to the beta version:
 
 - RuntimeClass resources must be recreated *after* upgrading to v1.14, and the
   `runtimeclasses.node.k8s.io` CRD should be manually deleted: