From 41114fd454d2a30b678f39875e156763e830e92d Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Wed, 16 Nov 2016 16:57:08 -0800 Subject: [PATCH 01/13] Initial commit of statefulsets concept. --- docs/concepts/index.md | 4 +- docs/concepts/object-metadata/statefulsets.md | 164 ++++++++++++++++++ 2 files changed, 167 insertions(+), 1 deletion(-) create mode 100644 docs/concepts/object-metadata/statefulsets.md diff --git a/docs/concepts/index.md b/docs/concepts/index.md index db56a3d05bd5a..5fc859fa41da2 100644 --- a/docs/concepts/index.md +++ b/docs/concepts/index.md @@ -5,7 +5,9 @@ The Concepts section of the Kubernetes documentation is a work in progress. #### Object Metadata -[Annotations](/docs/concepts/object-metadata/annotations/) +* [StatefulSets](/docs/concepts/object-metadata/statefulsets/) +* [Annotations](/docs/concepts/object-metadata/annotations/) + ### What's next diff --git a/docs/concepts/object-metadata/statefulsets.md b/docs/concepts/object-metadata/statefulsets.md new file mode 100644 index 0000000000000..31657c27333a5 --- /dev/null +++ b/docs/concepts/object-metadata/statefulsets.md @@ -0,0 +1,164 @@ +--- +--- + +{% capture overview %} +**StatefulSets are a beta feature in 1.5. This feature replaces the deprecated +PetSets feature from 1.4. Users of PetSets are referred to the 1.5 +[Upgrade Guide](/docs/task/upgrade-to-statefulset) +for further information on how to upgrade existing PetSets to StatefulSets.** + +A StatefulSet is a Controller that ensures that, at most, a given number of +replicas of a Pod are running at a time. Pods in a Stateful Set have an ordinal +(a unique integer index in the StatefulSet), a stable, unique network id that is +avialable in DNS, and stable, persistent storage. + +For a StatefulSet with N replicas, when Pods are being deployed, they are +created sequentially, in order from {0..N-1}. Before a new Pod is deployed, all +of its predecessors must be [Running and Ready](/docs/user-guide/pod-states). +When Pods are being deleted, they are terminated in reverse order, from {N-1..0}, +and no pod is terminated until its successors have been terminated and are +completely shutdown or its [Termination Grace Period](/docs/user-guide/pods/index#termination-of-pods)) +has elapsed. + +The exmpale below demonstrates the components of a StatefulSet. + +* A [Headless Service](/docs/user-guide/services/#headless-services), named nginx, is used to control the network domain. +* The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. +* The volumeClaimTemplates, will provide stable storage using [Persistent Volumes](/docs/user-guide/volumes/) provisioned by a + [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md). + +```yaml +--- +apiVersion: v1 +kind: Service +metadata: + name: nginx + labels: + app: nginx +spec: + ports: + - port: 80 + name: web + clusterIP: None + selector: + app: nginx +--- +apiVersion: apps/v1beta1 +kind: StatefulSet +metadata: + name: web +spec: + serviceName: "nginx" + replicas: 3 + template: + metadata: + labels: + app: nginx + annotations: + pod.alpha.kubernetes.io/initialized: "true" + spec: + terminationGracePeriodSeconds: 10 + containers: + - name: nginx + image: gcr.io/google_containers/nginx-slim:0.8 + ports: + - containerPort: 80 + name: web + volumeMounts: + - name: www + mountPath: /usr/share/nginx/html + volumeClaimTemplates: + - metadata: + name: www + annotations: + volume.alpha.kubernetes.io/storage-class: anything + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 1Gi +``` + +{% endcapture %} + +{% capture body %} +### When to Use a Stateful Set +StatefulSets are valuable for applications that require one or more of the +following. + +* Stable, unique network identifiers. +* Stable, persistent storage. +* Ordered, graceful deployment and scaling. +* Ordered, graceful deletion and termination. + +As it is generally easier to manage, if an application doesn't require any of +the above garuantees, and if it is feasible to do so, it should be deployed as +a set of stateless replicas. + +### Limitations +* StatefulSet is a beta resource, not available in any Kubernetes release prior to 1.5. +* As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver. +* The only updatable field on a StatefulSet is `replicas` +* The storage for a given pet must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. +* Deleting and/or scaling a StatefulSet down will *not* delete the volumes associated with the StatefulSet. This is done to ensure safety first, your data is more valuable than an auto purge of all related PetSet resources. **Deleting the Persistent Volume Claims will result in a deletion of the associated volumes**. +* All StatefulSets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the pets. The user is responsible for this Service. +* Updating an existing StatefulSet is currently a manual process, meaning you either need to deploy a new StatefulSet with the new image version, or orphan Pets one by one, update their image, and join them back to the cluster. + +### Pod Identity +StatefulSet Pods have a unique identity that is comprised of an ordinal, a +stable network identity, and stable storage. The identity sticks to the Pod, +regardless of which node it's (re) scheduled on. + +__Ordinal Index__ + +For a StatefulSet with N replicas, each Pod in the StatefulSet will be +assinged a integer ordinal, in the range [0,N), that is unique over the Set. + +__Stable Network Id__ + +The hostname of a Pod in a StatefulSet is derived from the name of the +StatefulSet and the ordinal of the Pod. The pattern for the constructed hostname +is `$(statefulset name)-$(ordinal)`. The example above will create three Pods +named `web-0,web-1,web-2`. +A StatelefulSet can use a [Headless Service](/docs/user-guide/services/#headless-services) +to control the domain of its Pods. The domain managed by this Service takes the form: +`$(service name).$(namespace).svc.cluster.local`, where "cluster.local" +is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). +As each Pod is created, it gets a matching DNS subdomain, taking the form: +`$(podname).$(governing service domain)`, where the governing service is defined +by the `serviceName` field on the StatefulSet. + +Here are some examples of choices for Cluster Domain, Service name, +StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods. + +Cluster Domain | Service (ns/name) | PetSet (ns/name) | PetSet Domain | Pet DNS | Pet Hostname | +-------------- | ----------------- | ----------------- | -------------- | ------- | ------------ | + cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} | + cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} | + kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} | + +Note that Cluster Domain will be set to `cluster.local` unless [otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). + +__Stable Storage__ + +[Persistent Volumes](/docs/user-guide/volumes/), one for each VolumeClaimTemplate, +are created based on the `volumeClaimTemplates` field of the StatefulSet. In the +example above, each Pod will recieve a single persistent volume with a storage +class of anything and 1 Gib of provisioned storage. When a Pod is (re)scheculed, +its volume(s) are avialable on the node on which it is launched. Note that, the +volumes associated with the Pods' Persistent Volume Claims are not deleted when +the Pods, or Stateful Set are deleted. This must be done manually. + +### Deployment and Scaling Garuantees +{% endcapture %} +When the exmample above is created, three Pods will be deployed in the order +web-0, web-1, web-2. web-1 will not be deployed before web-0 is +[Running and Ready](/docs/user-guide/pod-states), and web-2 will not be until +web-1 is Running and Ready. + +{% capture whatsnext %} +* Learn more about [this](...). +* See this [related task](...). +{% endcapture %} + +{% include templates/concept.md %} From f2e18d0288031327aedcb612bdc2caa3f3247182 Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Thu, 17 Nov 2016 13:00:33 -0800 Subject: [PATCH 02/13] adds sclaing and deployment section. fixes spelling errors. changes all StatefulSet referneces to Stateful Set and all PetSet references to Pet Set --- docs/concepts/index.md | 3 +- docs/concepts/object-metadata/statefulsets.md | 94 ++++++++++--------- 2 files changed, 53 insertions(+), 44 deletions(-) diff --git a/docs/concepts/index.md b/docs/concepts/index.md index 5fc859fa41da2..088708d125aed 100644 --- a/docs/concepts/index.md +++ b/docs/concepts/index.md @@ -5,8 +5,9 @@ The Concepts section of the Kubernetes documentation is a work in progress. #### Object Metadata -* [StatefulSets](/docs/concepts/object-metadata/statefulsets/) + * [Annotations](/docs/concepts/object-metadata/annotations/) +* [Stateful Sets](/docs/concepts/object-metadata/statefulsets/) ### What's next diff --git a/docs/concepts/object-metadata/statefulsets.md b/docs/concepts/object-metadata/statefulsets.md index 31657c27333a5..23f78aa2b811f 100644 --- a/docs/concepts/object-metadata/statefulsets.md +++ b/docs/concepts/object-metadata/statefulsets.md @@ -2,28 +2,28 @@ --- {% capture overview %} -**StatefulSets are a beta feature in 1.5. This feature replaces the deprecated -PetSets feature from 1.4. Users of PetSets are referred to the 1.5 -[Upgrade Guide](/docs/task/upgrade-to-statefulset) -for further information on how to upgrade existing PetSets to StatefulSets.** +**Stateful Sets are a beta feature in 1.5. This feature replaces the deprecated +Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5 +[Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) +for further information on how to upgrade existing Pet Sets to Stateful Sets.** -A StatefulSet is a Controller that ensures that, at most, a given number of +A Stateful Set is a Controller that ensures that, at most, a given number of replicas of a Pod are running at a time. Pods in a Stateful Set have an ordinal -(a unique integer index in the StatefulSet), a stable, unique network id that is -avialable in DNS, and stable, persistent storage. +(a unique integer index in the Stateful Set), a stable, unique network id that is +available in DNS, and stable, persistent storage. -For a StatefulSet with N replicas, when Pods are being deployed, they are +For a Stateful Set with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}. Before a new Pod is deployed, all of its predecessors must be [Running and Ready](/docs/user-guide/pod-states). When Pods are being deleted, they are terminated in reverse order, from {N-1..0}, and no pod is terminated until its successors have been terminated and are -completely shutdown or its [Termination Grace Period](/docs/user-guide/pods/index#termination-of-pods)) +completely shutdown or their[Termination Grace Periods](/docs/user-guide/pods/index#termination-of-pods) has elapsed. -The exmpale below demonstrates the components of a StatefulSet. +The example below demonstrates the components of a Stateful Set. * A [Headless Service](/docs/user-guide/services/#headless-services), named nginx, is used to control the network domain. -* The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. +* The Stateful Set, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. * The volumeClaimTemplates, will provide stable storage using [Persistent Volumes](/docs/user-guide/volumes/) provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md). @@ -83,7 +83,7 @@ spec: {% capture body %} ### When to Use a Stateful Set -StatefulSets are valuable for applications that require one or more of the +Stateful Sets are valuable for applications that require one or more of the following. * Stable, unique network identifiers. @@ -92,46 +92,46 @@ following. * Ordered, graceful deletion and termination. As it is generally easier to manage, if an application doesn't require any of -the above garuantees, and if it is feasible to do so, it should be deployed as +the above guarantees, and if it is feasible to do so, it should be deployed as a set of stateless replicas. ### Limitations -* StatefulSet is a beta resource, not available in any Kubernetes release prior to 1.5. +* Stateful Set is a beta resource, not available in any Kubernetes release prior to 1.5. * As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver. -* The only updatable field on a StatefulSet is `replicas` -* The storage for a given pet must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. -* Deleting and/or scaling a StatefulSet down will *not* delete the volumes associated with the StatefulSet. This is done to ensure safety first, your data is more valuable than an auto purge of all related PetSet resources. **Deleting the Persistent Volume Claims will result in a deletion of the associated volumes**. -* All StatefulSets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the pets. The user is responsible for this Service. -* Updating an existing StatefulSet is currently a manual process, meaning you either need to deploy a new StatefulSet with the new image version, or orphan Pets one by one, update their image, and join them back to the cluster. +* The only mutable field on a Stateful Set is `replicas` +* The storage for a given Pod must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. +* Deleting and/or scaling a Stateful Set down will *not* delete the volumes associated with the Stateful Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Stateful Set resources. **Deleting the Persistent Volume Claims will result in a deletion of the associated volumes**. +* Stateful Sets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. +* Updating an existing Stateful Set is currently a manual process, meaning you either need to deploy a new Stateful Set with the new image version, or orphan Pods one by one, update their image, and join them back to the cluster. ### Pod Identity -StatefulSet Pods have a unique identity that is comprised of an ordinal, a +Stateful Set Pods have a unique identity that is comprised of an ordinal, a stable network identity, and stable storage. The identity sticks to the Pod, regardless of which node it's (re) scheduled on. __Ordinal Index__ -For a StatefulSet with N replicas, each Pod in the StatefulSet will be -assinged a integer ordinal, in the range [0,N), that is unique over the Set. +For a Stateful Set with N replicas, each Pod in the Stateful Set will be +assigned a integer ordinal, in the range [0,N), that is unique over the Set. __Stable Network Id__ -The hostname of a Pod in a StatefulSet is derived from the name of the -StatefulSet and the ordinal of the Pod. The pattern for the constructed hostname +The hostname of a Pod in a Stateful Set is derived from the name of the +Stateful Set and the ordinal of the Pod. The pattern for the constructed hostname is `$(statefulset name)-$(ordinal)`. The example above will create three Pods named `web-0,web-1,web-2`. -A StatelefulSet can use a [Headless Service](/docs/user-guide/services/#headless-services) +A Stateful Set can use a [Headless Service](/docs/user-guide/services/#headless-services) to control the domain of its Pods. The domain managed by this Service takes the form: `$(service name).$(namespace).svc.cluster.local`, where "cluster.local" is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). As each Pod is created, it gets a matching DNS subdomain, taking the form: `$(podname).$(governing service domain)`, where the governing service is defined -by the `serviceName` field on the StatefulSet. +by the `serviceName` field on the Stateful Set. Here are some examples of choices for Cluster Domain, Service name, -StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods. +Stateful Set name, and how that affects the DNS names for the Stateful Set's Pods. -Cluster Domain | Service (ns/name) | PetSet (ns/name) | PetSet Domain | Pet DNS | Pet Hostname | +Cluster Domain | Service (ns/name) | Stateful Set (ns/name) | Stateful Set Domain | Pod DNS | Pod Hostname | -------------- | ----------------- | ----------------- | -------------- | ------- | ------------ | cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} | cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} | @@ -141,24 +141,32 @@ Note that Cluster Domain will be set to `cluster.local` unless [otherwise config __Stable Storage__ -[Persistent Volumes](/docs/user-guide/volumes/), one for each VolumeClaimTemplate, -are created based on the `volumeClaimTemplates` field of the StatefulSet. In the -example above, each Pod will recieve a single persistent volume with a storage -class of anything and 1 Gib of provisioned storage. When a Pod is (re)scheculed, -its volume(s) are avialable on the node on which it is launched. Note that, the +[Persistent Volumes](/docs/user-guide/volumes/), one for each Volume Claim Template, +are created based on the `volumeClaimTemplates` field of the Stateful Set. In the +example above, each Pod will receive a single persistent volume with a storage +class of anything and 1 Gib of provisioned storage. When a Pod is (re)scheduled, +its volume(s) are available on the node on which it is launched. Note that, the volumes associated with the Pods' Persistent Volume Claims are not deleted when the Pods, or Stateful Set are deleted. This must be done manually. -### Deployment and Scaling Garuantees -{% endcapture %} -When the exmample above is created, three Pods will be deployed in the order -web-0, web-1, web-2. web-1 will not be deployed before web-0 is -[Running and Ready](/docs/user-guide/pod-states), and web-2 will not be until -web-1 is Running and Ready. +### Deployment and Scaling Guarantee -{% capture whatsnext %} -* Learn more about [this](...). -* See this [related task](...). -{% endcapture %} +* For a Stateful Set with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}. +* When Pods are being deleted, they are terminated in reverse order, from {N-1..0}. +* Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready. +* Before a Pod is terminated, all of its successors must be completely shutdown. +When the web example above is created, three Pods will be deployed in the order +web-0, web-1, web-2. web-1 will not be deployed before web-0 is +[Running and Ready](/docs/user-guide/pod-states), and web-2 will not be deployed until +web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before +web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and +becomes Running and Ready. + +If a user were to scale the deployed example by patching the Stateful Set such that +`replicas=1`, web-2 would be terminated first. web-1 would not be terminated until web-2 +is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and +is completely shutdown, but prior to web-1's termination, web-1 would not be terminated +until web-0 is Running and Ready. +{% endcapture %} {% include templates/concept.md %} From d6ea2b291c071aecb99fda4e9b9ccafa5a806817 Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Thu, 17 Nov 2016 15:05:36 -0800 Subject: [PATCH 03/13] updates directory structure to add controllers directory moves statefulset.md into controllers update _data/concepts.yml to show a TOC based on the above --- _data/concepts.yml | 4 ++++ docs/concepts/index.md | 2 +- docs/concepts/object-metadata/statefulsets.md | 2 +- 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/_data/concepts.yml b/_data/concepts.yml index f9422daa988a6..fc2b8beb7d349 100644 --- a/_data/concepts.yml +++ b/_data/concepts.yml @@ -7,3 +7,7 @@ toc: section: - title: Annotations path: /docs/concepts/object-metadata/annotations/ +- title: Controllers + section: + - title: Stateful Sets + path: /docs/concepts/controllers/statefulsets/ diff --git a/docs/concepts/index.md b/docs/concepts/index.md index 088708d125aed..c35b9b6d0ff24 100644 --- a/docs/concepts/index.md +++ b/docs/concepts/index.md @@ -7,7 +7,7 @@ The Concepts section of the Kubernetes documentation is a work in progress. * [Annotations](/docs/concepts/object-metadata/annotations/) -* [Stateful Sets](/docs/concepts/object-metadata/statefulsets/) +* [Stateful Sets](/docs/concepts/controllers/statefulsets/) ### What's next diff --git a/docs/concepts/object-metadata/statefulsets.md b/docs/concepts/object-metadata/statefulsets.md index 23f78aa2b811f..99d0228bca132 100644 --- a/docs/concepts/object-metadata/statefulsets.md +++ b/docs/concepts/object-metadata/statefulsets.md @@ -4,7 +4,7 @@ {% capture overview %} **Stateful Sets are a beta feature in 1.5. This feature replaces the deprecated Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5 -[Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) +[Upgrade Guide](docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md) for further information on how to upgrade existing Pet Sets to Stateful Sets.** A Stateful Set is a Controller that ensures that, at most, a given number of From 8f0e37b24c9a22f321574ecf09b628b70d9d2c07 Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Thu, 17 Nov 2016 15:23:07 -0800 Subject: [PATCH 04/13] completes move of statefulset to controllers shortens intro in favor of another Components section --- .../statefulsets.md | 65 ++++++++----------- 1 file changed, 28 insertions(+), 37 deletions(-) rename docs/concepts/{object-metadata => controllers}/statefulsets.md (89%) diff --git a/docs/concepts/object-metadata/statefulsets.md b/docs/concepts/controllers/statefulsets.md similarity index 89% rename from docs/concepts/object-metadata/statefulsets.md rename to docs/concepts/controllers/statefulsets.md index 99d0228bca132..8818241f46297 100644 --- a/docs/concepts/object-metadata/statefulsets.md +++ b/docs/concepts/controllers/statefulsets.md @@ -4,22 +4,38 @@ {% capture overview %} **Stateful Sets are a beta feature in 1.5. This feature replaces the deprecated Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5 -[Upgrade Guide](docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md) +[Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) for further information on how to upgrade existing Pet Sets to Stateful Sets.** -A Stateful Set is a Controller that ensures that, at most, a given number of -replicas of a Pod are running at a time. Pods in a Stateful Set have an ordinal -(a unique integer index in the Stateful Set), a stable, unique network id that is -available in DNS, and stable, persistent storage. +A Stateful Set is a Controller that ensures that provides a unique identity to +its Pods, and provides garuantees about the ordering of deployment and scaling. +{% endcapture %} -For a Stateful Set with N replicas, when Pods are being deployed, they are -created sequentially, in order from {0..N-1}. Before a new Pod is deployed, all -of its predecessors must be [Running and Ready](/docs/user-guide/pod-states). -When Pods are being deleted, they are terminated in reverse order, from {N-1..0}, -and no pod is terminated until its successors have been terminated and are -completely shutdown or their[Termination Grace Periods](/docs/user-guide/pods/index#termination-of-pods) -has elapsed. +{% capture body %} +### When to Use a Stateful Set +Stateful Sets are valuable for applications that require one or more of the +following. + +* Stable, unique network identifiers. +* Stable, persistent storage. +* Ordered, graceful deployment and scaling. +* Ordered, graceful deletion and termination. + +As it is generally easier to manage, if an application doesn't require any of +the above guarantees, and if it is feasible to do so, it should be deployed as +a set of stateless replicas. + +### Limitations +* Stateful Set is a beta resource, not available in any Kubernetes release prior to 1.5. +* As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver. +* The only mutable field on a Stateful Set is `replicas` +* The storage for a given Pod must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. +* Deleting and/or scaling a Stateful Set down will *not* delete the volumes associated with the Stateful Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Stateful Set resources. **Deleting the Persistent Volume Claims will result in a deletion of the associated volumes**. +* Stateful Sets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. +* Updating an existing Stateful Set is currently a manual process, meaning you either need to deploy a new Stateful Set with the new image version, or orphan Pods one by one, update their image, and join them back to the cluster. + +### Components The example below demonstrates the components of a Stateful Set. * A [Headless Service](/docs/user-guide/services/#headless-services), named nginx, is used to control the network domain. @@ -79,31 +95,6 @@ spec: storage: 1Gi ``` -{% endcapture %} - -{% capture body %} -### When to Use a Stateful Set -Stateful Sets are valuable for applications that require one or more of the -following. - -* Stable, unique network identifiers. -* Stable, persistent storage. -* Ordered, graceful deployment and scaling. -* Ordered, graceful deletion and termination. - -As it is generally easier to manage, if an application doesn't require any of -the above guarantees, and if it is feasible to do so, it should be deployed as -a set of stateless replicas. - -### Limitations -* Stateful Set is a beta resource, not available in any Kubernetes release prior to 1.5. -* As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver. -* The only mutable field on a Stateful Set is `replicas` -* The storage for a given Pod must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. -* Deleting and/or scaling a Stateful Set down will *not* delete the volumes associated with the Stateful Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Stateful Set resources. **Deleting the Persistent Volume Claims will result in a deletion of the associated volumes**. -* Stateful Sets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. -* Updating an existing Stateful Set is currently a manual process, meaning you either need to deploy a new Stateful Set with the new image version, or orphan Pods one by one, update their image, and join them back to the cluster. - ### Pod Identity Stateful Set Pods have a unique identity that is comprised of an ordinal, a stable network identity, and stable storage. The identity sticks to the Pod, From 26f0e6e7a68b1dbe17f9790cb8913a52d85a7b0a Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Fri, 18 Nov 2016 10:13:23 -0800 Subject: [PATCH 05/13] removes legacy claim about mutability of stateful set --- docs/concepts/controllers/statefulsets.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/concepts/controllers/statefulsets.md b/docs/concepts/controllers/statefulsets.md index 8818241f46297..3b59025f463c0 100644 --- a/docs/concepts/controllers/statefulsets.md +++ b/docs/concepts/controllers/statefulsets.md @@ -29,7 +29,6 @@ a set of stateless replicas. ### Limitations * Stateful Set is a beta resource, not available in any Kubernetes release prior to 1.5. * As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver. -* The only mutable field on a Stateful Set is `replicas` * The storage for a given Pod must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. * Deleting and/or scaling a Stateful Set down will *not* delete the volumes associated with the Stateful Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Stateful Set resources. **Deleting the Persistent Volume Claims will result in a deletion of the associated volumes**. * Stateful Sets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. From 7aa8c5d2e6253739a2da144d933b9daa1804ec36 Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Fri, 18 Nov 2016 10:26:54 -0800 Subject: [PATCH 06/13] upgrades the version of storage class to beta adds assignees clarifies stable storage description --- docs/concepts/controllers/statefulsets.md | 24 +++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/docs/concepts/controllers/statefulsets.md b/docs/concepts/controllers/statefulsets.md index 3b59025f463c0..90dab101e30cd 100644 --- a/docs/concepts/controllers/statefulsets.md +++ b/docs/concepts/controllers/statefulsets.md @@ -1,4 +1,12 @@ --- +assignees: +- bprashanth +- enisoc +- erictune +- foxish +- janetkuo +- kow3ns +- smarterclayton --- {% capture overview %} @@ -69,8 +77,6 @@ spec: metadata: labels: app: nginx - annotations: - pod.alpha.kubernetes.io/initialized: "true" spec: terminationGracePeriodSeconds: 10 containers: @@ -86,7 +92,7 @@ spec: - metadata: name: www annotations: - volume.alpha.kubernetes.io/storage-class: anything + volume.beta.kubernetes.io/storage-class: anything spec: accessModes: [ "ReadWriteOnce" ] resources: @@ -127,17 +133,19 @@ Cluster Domain | Service (ns/name) | Stateful Set (ns/name) | Stateful Set Doma cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} | kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} | -Note that Cluster Domain will be set to `cluster.local` unless [otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). +Note that Cluster Domain will be set to `cluster.local` unless +[otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). __Stable Storage__ [Persistent Volumes](/docs/user-guide/volumes/), one for each Volume Claim Template, are created based on the `volumeClaimTemplates` field of the Stateful Set. In the example above, each Pod will receive a single persistent volume with a storage -class of anything and 1 Gib of provisioned storage. When a Pod is (re)scheduled, -its volume(s) are available on the node on which it is launched. Note that, the -volumes associated with the Pods' Persistent Volume Claims are not deleted when -the Pods, or Stateful Set are deleted. This must be done manually. +class of anything and 1 Gib of provisioned storage. When a Pod is (re) scheduled onto +a node, its `volumeMounts` mount the Persistent Volumes associated with its +Persistent Volume Claims. That is, the Note that, the Persistent Volumes associated with the +Pods' Persistent Volume Claims are not deleted when the Pods, or Stateful Set are deleted. +This must be done manually. ### Deployment and Scaling Guarantee From 094a51fc2e7413c37e851a74bde1edf93bcc0438 Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Fri, 18 Nov 2016 15:35:14 -0800 Subject: [PATCH 07/13] Fixes typo in guarantees Removes deprecation from comment wrt PetSets Creates separate section in concepts/index.md for Controllers Fixes grammer "an integer" Removes claim about persistent storage being auto-deleted Clarifies stability Adds call out tick around attributes --- docs/concepts/controllers/statefulsets.md | 17 +++++++++-------- docs/concepts/index.md | 2 ++ 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/concepts/controllers/statefulsets.md b/docs/concepts/controllers/statefulsets.md index 90dab101e30cd..dab22b8b6fd2b 100644 --- a/docs/concepts/controllers/statefulsets.md +++ b/docs/concepts/controllers/statefulsets.md @@ -10,13 +10,13 @@ assignees: --- {% capture overview %} -**Stateful Sets are a beta feature in 1.5. This feature replaces the deprecated +**Stateful Sets are a beta feature in 1.5. This feature replaces the Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5 [Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) for further information on how to upgrade existing Pet Sets to Stateful Sets.** -A Stateful Set is a Controller that ensures that provides a unique identity to -its Pods, and provides garuantees about the ordering of deployment and scaling. +A Stateful Set is a Controlle provides a unique identity to its Pods, and it provides +guarantees about the ordering of deployment and scaling. {% endcapture %} {% capture body %} @@ -30,6 +30,7 @@ following. * Ordered, graceful deployment and scaling. * Ordered, graceful deletion and termination. +In the above, by stable, we mean persistent across Pod (re) schedulings. As it is generally easier to manage, if an application doesn't require any of the above guarantees, and if it is feasible to do so, it should be deployed as a set of stateless replicas. @@ -38,7 +39,7 @@ a set of stateless replicas. * Stateful Set is a beta resource, not available in any Kubernetes release prior to 1.5. * As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver. * The storage for a given Pod must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. -* Deleting and/or scaling a Stateful Set down will *not* delete the volumes associated with the Stateful Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Stateful Set resources. **Deleting the Persistent Volume Claims will result in a deletion of the associated volumes**. +* Deleting and/or scaling a Stateful Set down will *not* delete the volumes associated with the Stateful Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Stateful Set resources. * Stateful Sets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. * Updating an existing Stateful Set is currently a manual process, meaning you either need to deploy a new Stateful Set with the new image version, or orphan Pods one by one, update their image, and join them back to the cluster. @@ -108,9 +109,9 @@ regardless of which node it's (re) scheduled on. __Ordinal Index__ For a Stateful Set with N replicas, each Pod in the Stateful Set will be -assigned a integer ordinal, in the range [0,N), that is unique over the Set. +assigned an integer ordinal, in the range [0,N), that is unique over the Set. -__Stable Network Id__ +__Stable Network ID__ The hostname of a Pod in a Stateful Set is derived from the name of the Stateful Set and the ordinal of the Pod. The pattern for the constructed hostname @@ -141,9 +142,9 @@ __Stable Storage__ [Persistent Volumes](/docs/user-guide/volumes/), one for each Volume Claim Template, are created based on the `volumeClaimTemplates` field of the Stateful Set. In the example above, each Pod will receive a single persistent volume with a storage -class of anything and 1 Gib of provisioned storage. When a Pod is (re) scheduled onto +class of `anything` and 1 Gib of provisioned storage. When a Pod is (re) scheduled onto a node, its `volumeMounts` mount the Persistent Volumes associated with its -Persistent Volume Claims. That is, the Note that, the Persistent Volumes associated with the +Persistent Volume Claims. Note that, the Persistent Volumes associated with the Pods' Persistent Volume Claims are not deleted when the Pods, or Stateful Set are deleted. This must be done manually. diff --git a/docs/concepts/index.md b/docs/concepts/index.md index c35b9b6d0ff24..4428d38ccda17 100644 --- a/docs/concepts/index.md +++ b/docs/concepts/index.md @@ -7,6 +7,8 @@ The Concepts section of the Kubernetes documentation is a work in progress. * [Annotations](/docs/concepts/object-metadata/annotations/) + +#### Controllers * [Stateful Sets](/docs/concepts/controllers/statefulsets/) From 6e65abe9e514279b11055c62fa1f7328e7142e6c Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Fri, 18 Nov 2016 15:43:42 -0800 Subject: [PATCH 08/13] Fixes typo --- docs/concepts/controllers/statefulsets.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/controllers/statefulsets.md b/docs/concepts/controllers/statefulsets.md index dab22b8b6fd2b..1cd9c51f10206 100644 --- a/docs/concepts/controllers/statefulsets.md +++ b/docs/concepts/controllers/statefulsets.md @@ -15,7 +15,7 @@ Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5 [Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) for further information on how to upgrade existing Pet Sets to Stateful Sets.** -A Stateful Set is a Controlle provides a unique identity to its Pods, and it provides +A Stateful Set is a Controller provides a unique identity to its Pods, and it provides guarantees about the ordering of deployment and scaling. {% endcapture %} From 77dc8f96b3fb114f28e5f31e81c20f6828fc8a8f Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Tue, 22 Nov 2016 15:50:18 -0800 Subject: [PATCH 09/13] Changes introduction wording Removes storage annotation --- docs/concepts/controllers/statefulsets.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/concepts/controllers/statefulsets.md b/docs/concepts/controllers/statefulsets.md index 1cd9c51f10206..7ee7cacc49468 100644 --- a/docs/concepts/controllers/statefulsets.md +++ b/docs/concepts/controllers/statefulsets.md @@ -15,7 +15,7 @@ Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5 [Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) for further information on how to upgrade existing Pet Sets to Stateful Sets.** -A Stateful Set is a Controller provides a unique identity to its Pods, and it provides +A Stateful Set is a Controller that provides a unique identity to its Pods. It provides guarantees about the ordering of deployment and scaling. {% endcapture %} @@ -92,8 +92,6 @@ spec: volumeClaimTemplates: - metadata: name: www - annotations: - volume.beta.kubernetes.io/storage-class: anything spec: accessModes: [ "ReadWriteOnce" ] resources: From 92f8d24d479ebb4aba2349f7380e7535fe57669b Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Mon, 28 Nov 2016 16:32:19 -0800 Subject: [PATCH 10/13] Gives examples of Deployment and ReplicaSet as Stateful Set alternative Removes redundant hyperlinks Makes statement about updates limitation more terse --- docs/concepts/controllers/statefulsets.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/concepts/controllers/statefulsets.md b/docs/concepts/controllers/statefulsets.md index 7ee7cacc49468..bbd8ae614fc82 100644 --- a/docs/concepts/controllers/statefulsets.md +++ b/docs/concepts/controllers/statefulsets.md @@ -33,7 +33,9 @@ following. In the above, by stable, we mean persistent across Pod (re) schedulings. As it is generally easier to manage, if an application doesn't require any of the above guarantees, and if it is feasible to do so, it should be deployed as -a set of stateless replicas. +a set of stateless replicas. Before deciding to use a Stateful Set, you should +be certain that a [Deployment](/docs/user-guide/deployments/) or a +[Replica Set](/docs/user-guide/replicasets/) is not better suited to your needs. ### Limitations * Stateful Set is a beta resource, not available in any Kubernetes release prior to 1.5. @@ -41,15 +43,15 @@ a set of stateless replicas. * The storage for a given Pod must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. * Deleting and/or scaling a Stateful Set down will *not* delete the volumes associated with the Stateful Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Stateful Set resources. * Stateful Sets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. -* Updating an existing Stateful Set is currently a manual process, meaning you either need to deploy a new Stateful Set with the new image version, or orphan Pods one by one, update their image, and join them back to the cluster. +* Updating an existing Stateful Set is currently a manual process. ### Components The example below demonstrates the components of a Stateful Set. -* A [Headless Service](/docs/user-guide/services/#headless-services), named nginx, is used to control the network domain. +* A Headless Service, named nginx, is used to control the network domain. * The Stateful Set, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. * The volumeClaimTemplates, will provide stable storage using [Persistent Volumes](/docs/user-guide/volumes/) provisioned by a - [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md). + Persistent Volume Provisioner. ```yaml --- From 349e41f634bc3b001abff031b74d4981468bb0b0 Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Tue, 29 Nov 2016 09:55:02 -0800 Subject: [PATCH 11/13] Changes all API Objects to CamelCase (e.g Stateful Set becomes StatefulSet and Persistent Volume becomes PersistentVolume) --- _data/concepts.yml | 2 +- docs/concepts/controllers/statefulsets.md | 64 +++++++++++------------ docs/concepts/index.md | 2 +- 3 files changed, 34 insertions(+), 34 deletions(-) diff --git a/_data/concepts.yml b/_data/concepts.yml index fc2b8beb7d349..36f2e591d9903 100644 --- a/_data/concepts.yml +++ b/_data/concepts.yml @@ -9,5 +9,5 @@ toc: path: /docs/concepts/object-metadata/annotations/ - title: Controllers section: - - title: Stateful Sets + - title: StatefulSets path: /docs/concepts/controllers/statefulsets/ diff --git a/docs/concepts/controllers/statefulsets.md b/docs/concepts/controllers/statefulsets.md index bbd8ae614fc82..1555eeb52ff06 100644 --- a/docs/concepts/controllers/statefulsets.md +++ b/docs/concepts/controllers/statefulsets.md @@ -10,19 +10,19 @@ assignees: --- {% capture overview %} -**Stateful Sets are a beta feature in 1.5. This feature replaces the +**StatefulSets are a beta feature in 1.5. This feature replaces the Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5 [Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) -for further information on how to upgrade existing Pet Sets to Stateful Sets.** +for further information on how to upgrade existing Pet Sets to StatefulSets.** -A Stateful Set is a Controller that provides a unique identity to its Pods. It provides +A StatefulSet is a Controller that provides a unique identity to its Pods. It provides guarantees about the ordering of deployment and scaling. {% endcapture %} {% capture body %} -### When to Use a Stateful Set -Stateful Sets are valuable for applications that require one or more of the +### When to Use a StatefulSet +StatefulSets are valuable for applications that require one or more of the following. * Stable, unique network identifiers. @@ -33,25 +33,25 @@ following. In the above, by stable, we mean persistent across Pod (re) schedulings. As it is generally easier to manage, if an application doesn't require any of the above guarantees, and if it is feasible to do so, it should be deployed as -a set of stateless replicas. Before deciding to use a Stateful Set, you should +a set of stateless replicas. Before deciding to use a StatefulSet, you should be certain that a [Deployment](/docs/user-guide/deployments/) or a -[Replica Set](/docs/user-guide/replicasets/) is not better suited to your needs. +[ReplicaSet](/docs/user-guide/replicasets/) is not better suited to your needs. ### Limitations -* Stateful Set is a beta resource, not available in any Kubernetes release prior to 1.5. +* StatefulSet is a beta resource, not available in any Kubernetes release prior to 1.5. * As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver. -* The storage for a given Pod must either be provisioned by a [Persistent Volume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. -* Deleting and/or scaling a Stateful Set down will *not* delete the volumes associated with the Stateful Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Stateful Set resources. -* Stateful Sets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. -* Updating an existing Stateful Set is currently a manual process. +* The storage for a given Pod must either be provisioned by a [PersistentVolume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. +* Deleting and/or scaling a StatefulSet down will *not* delete the volumes associated with the StatefulSet. This is done to ensure safety first, your data is more valuable than an auto purge of all related StatefulSet resources. +* StatefulSets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. +* Updating an existing StatefulSet is currently a manual process. ### Components -The example below demonstrates the components of a Stateful Set. +The example below demonstrates the components of a StatefulSet. * A Headless Service, named nginx, is used to control the network domain. -* The Stateful Set, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. -* The volumeClaimTemplates, will provide stable storage using [Persistent Volumes](/docs/user-guide/volumes/) provisioned by a - Persistent Volume Provisioner. +* The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. +* The volumeClaimTemplates, will provide stable storage using [PersistentVolumes](/docs/user-guide/volumes/) provisioned by a + PersistentVolume Provisioner. ```yaml --- @@ -102,33 +102,33 @@ spec: ``` ### Pod Identity -Stateful Set Pods have a unique identity that is comprised of an ordinal, a +StatefulSet Pods have a unique identity that is comprised of an ordinal, a stable network identity, and stable storage. The identity sticks to the Pod, regardless of which node it's (re) scheduled on. __Ordinal Index__ -For a Stateful Set with N replicas, each Pod in the Stateful Set will be +For a StatefulSet with N replicas, each Pod in the StatefulSet will be assigned an integer ordinal, in the range [0,N), that is unique over the Set. __Stable Network ID__ -The hostname of a Pod in a Stateful Set is derived from the name of the -Stateful Set and the ordinal of the Pod. The pattern for the constructed hostname +The hostname of a Pod in a StatefulSet is derived from the name of the +StatefulSet and the ordinal of the Pod. The pattern for the constructed hostname is `$(statefulset name)-$(ordinal)`. The example above will create three Pods named `web-0,web-1,web-2`. -A Stateful Set can use a [Headless Service](/docs/user-guide/services/#headless-services) +A StatefulSet can use a [Headless Service](/docs/user-guide/services/#headless-services) to control the domain of its Pods. The domain managed by this Service takes the form: `$(service name).$(namespace).svc.cluster.local`, where "cluster.local" is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). As each Pod is created, it gets a matching DNS subdomain, taking the form: `$(podname).$(governing service domain)`, where the governing service is defined -by the `serviceName` field on the Stateful Set. +by the `serviceName` field on the StatefulSet. Here are some examples of choices for Cluster Domain, Service name, -Stateful Set name, and how that affects the DNS names for the Stateful Set's Pods. +StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods. -Cluster Domain | Service (ns/name) | Stateful Set (ns/name) | Stateful Set Domain | Pod DNS | Pod Hostname | +Cluster Domain | Service (ns/name) | StatefulSet (ns/name) | StatefulSet Domain | Pod DNS | Pod Hostname | -------------- | ----------------- | ----------------- | -------------- | ------- | ------------ | cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} | cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} | @@ -139,18 +139,18 @@ Note that Cluster Domain will be set to `cluster.local` unless __Stable Storage__ -[Persistent Volumes](/docs/user-guide/volumes/), one for each Volume Claim Template, -are created based on the `volumeClaimTemplates` field of the Stateful Set. In the -example above, each Pod will receive a single persistent volume with a storage +[PersistentVolumes](/docs/user-guide/volumes/), one for each Volume Claim Template, +are created based on the `volumeClaimTemplates` field of the StatefulSet. In the +example above, each Pod will receive a single PersistentVolume with a storage class of `anything` and 1 Gib of provisioned storage. When a Pod is (re) scheduled onto -a node, its `volumeMounts` mount the Persistent Volumes associated with its -Persistent Volume Claims. Note that, the Persistent Volumes associated with the -Pods' Persistent Volume Claims are not deleted when the Pods, or Stateful Set are deleted. +a node, its `volumeMounts` mount the PersistentVolumes associated with its +PersistentVolume Claims. Note that, the PersistentVolumes associated with the +Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted. This must be done manually. ### Deployment and Scaling Guarantee -* For a Stateful Set with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}. +* For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}. * When Pods are being deleted, they are terminated in reverse order, from {N-1..0}. * Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready. * Before a Pod is terminated, all of its successors must be completely shutdown. @@ -162,7 +162,7 @@ web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Rea web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and becomes Running and Ready. -If a user were to scale the deployed example by patching the Stateful Set such that +If a user were to scale the deployed example by patching the StatefulSet such that `replicas=1`, web-2 would be terminated first. web-1 would not be terminated until web-2 is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and is completely shutdown, but prior to web-1's termination, web-1 would not be terminated diff --git a/docs/concepts/index.md b/docs/concepts/index.md index 4428d38ccda17..b6e6d89a0b320 100644 --- a/docs/concepts/index.md +++ b/docs/concepts/index.md @@ -9,7 +9,7 @@ The Concepts section of the Kubernetes documentation is a work in progress. * [Annotations](/docs/concepts/object-metadata/annotations/) #### Controllers -* [Stateful Sets](/docs/concepts/controllers/statefulsets/) +* [StatefulSets](/docs/concepts/controllers/statefulsets/) ### What's next From 1bc1017d752377f01cf995007ca8c4e0d0ca6911 Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Fri, 2 Dec 2016 13:51:38 -0800 Subject: [PATCH 12/13] Address editorial comments --- _data/concepts.yml | 2 +- docs/concepts/controllers/statefulsets.md | 171 ---------------------- docs/concepts/index.md | 2 +- 3 files changed, 2 insertions(+), 173 deletions(-) delete mode 100644 docs/concepts/controllers/statefulsets.md diff --git a/_data/concepts.yml b/_data/concepts.yml index 36f2e591d9903..93c790c0ab61f 100644 --- a/_data/concepts.yml +++ b/_data/concepts.yml @@ -10,4 +10,4 @@ toc: - title: Controllers section: - title: StatefulSets - path: /docs/concepts/controllers/statefulsets/ + path: /docs/concepts/abstractions/controllers/statefulsets/ diff --git a/docs/concepts/controllers/statefulsets.md b/docs/concepts/controllers/statefulsets.md deleted file mode 100644 index 1555eeb52ff06..0000000000000 --- a/docs/concepts/controllers/statefulsets.md +++ /dev/null @@ -1,171 +0,0 @@ ---- -assignees: -- bprashanth -- enisoc -- erictune -- foxish -- janetkuo -- kow3ns -- smarterclayton ---- - -{% capture overview %} -**StatefulSets are a beta feature in 1.5. This feature replaces the -Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5 -[Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) -for further information on how to upgrade existing Pet Sets to StatefulSets.** - -A StatefulSet is a Controller that provides a unique identity to its Pods. It provides -guarantees about the ordering of deployment and scaling. -{% endcapture %} - -{% capture body %} - -### When to Use a StatefulSet -StatefulSets are valuable for applications that require one or more of the -following. - -* Stable, unique network identifiers. -* Stable, persistent storage. -* Ordered, graceful deployment and scaling. -* Ordered, graceful deletion and termination. - -In the above, by stable, we mean persistent across Pod (re) schedulings. -As it is generally easier to manage, if an application doesn't require any of -the above guarantees, and if it is feasible to do so, it should be deployed as -a set of stateless replicas. Before deciding to use a StatefulSet, you should -be certain that a [Deployment](/docs/user-guide/deployments/) or a -[ReplicaSet](/docs/user-guide/replicasets/) is not better suited to your needs. - -### Limitations -* StatefulSet is a beta resource, not available in any Kubernetes release prior to 1.5. -* As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver. -* The storage for a given Pod must either be provisioned by a [PersistentVolume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. -* Deleting and/or scaling a StatefulSet down will *not* delete the volumes associated with the StatefulSet. This is done to ensure safety first, your data is more valuable than an auto purge of all related StatefulSet resources. -* StatefulSets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service. -* Updating an existing StatefulSet is currently a manual process. - -### Components -The example below demonstrates the components of a StatefulSet. - -* A Headless Service, named nginx, is used to control the network domain. -* The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. -* The volumeClaimTemplates, will provide stable storage using [PersistentVolumes](/docs/user-guide/volumes/) provisioned by a - PersistentVolume Provisioner. - -```yaml ---- -apiVersion: v1 -kind: Service -metadata: - name: nginx - labels: - app: nginx -spec: - ports: - - port: 80 - name: web - clusterIP: None - selector: - app: nginx ---- -apiVersion: apps/v1beta1 -kind: StatefulSet -metadata: - name: web -spec: - serviceName: "nginx" - replicas: 3 - template: - metadata: - labels: - app: nginx - spec: - terminationGracePeriodSeconds: 10 - containers: - - name: nginx - image: gcr.io/google_containers/nginx-slim:0.8 - ports: - - containerPort: 80 - name: web - volumeMounts: - - name: www - mountPath: /usr/share/nginx/html - volumeClaimTemplates: - - metadata: - name: www - spec: - accessModes: [ "ReadWriteOnce" ] - resources: - requests: - storage: 1Gi -``` - -### Pod Identity -StatefulSet Pods have a unique identity that is comprised of an ordinal, a -stable network identity, and stable storage. The identity sticks to the Pod, -regardless of which node it's (re) scheduled on. - -__Ordinal Index__ - -For a StatefulSet with N replicas, each Pod in the StatefulSet will be -assigned an integer ordinal, in the range [0,N), that is unique over the Set. - -__Stable Network ID__ - -The hostname of a Pod in a StatefulSet is derived from the name of the -StatefulSet and the ordinal of the Pod. The pattern for the constructed hostname -is `$(statefulset name)-$(ordinal)`. The example above will create three Pods -named `web-0,web-1,web-2`. -A StatefulSet can use a [Headless Service](/docs/user-guide/services/#headless-services) -to control the domain of its Pods. The domain managed by this Service takes the form: -`$(service name).$(namespace).svc.cluster.local`, where "cluster.local" -is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). -As each Pod is created, it gets a matching DNS subdomain, taking the form: -`$(podname).$(governing service domain)`, where the governing service is defined -by the `serviceName` field on the StatefulSet. - -Here are some examples of choices for Cluster Domain, Service name, -StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods. - -Cluster Domain | Service (ns/name) | StatefulSet (ns/name) | StatefulSet Domain | Pod DNS | Pod Hostname | --------------- | ----------------- | ----------------- | -------------- | ------- | ------------ | - cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} | - cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} | - kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} | - -Note that Cluster Domain will be set to `cluster.local` unless -[otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). - -__Stable Storage__ - -[PersistentVolumes](/docs/user-guide/volumes/), one for each Volume Claim Template, -are created based on the `volumeClaimTemplates` field of the StatefulSet. In the -example above, each Pod will receive a single PersistentVolume with a storage -class of `anything` and 1 Gib of provisioned storage. When a Pod is (re) scheduled onto -a node, its `volumeMounts` mount the PersistentVolumes associated with its -PersistentVolume Claims. Note that, the PersistentVolumes associated with the -Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted. -This must be done manually. - -### Deployment and Scaling Guarantee - -* For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}. -* When Pods are being deleted, they are terminated in reverse order, from {N-1..0}. -* Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready. -* Before a Pod is terminated, all of its successors must be completely shutdown. - -When the web example above is created, three Pods will be deployed in the order -web-0, web-1, web-2. web-1 will not be deployed before web-0 is -[Running and Ready](/docs/user-guide/pod-states), and web-2 will not be deployed until -web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before -web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and -becomes Running and Ready. - -If a user were to scale the deployed example by patching the StatefulSet such that -`replicas=1`, web-2 would be terminated first. web-1 would not be terminated until web-2 -is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and -is completely shutdown, but prior to web-1's termination, web-1 would not be terminated -until web-0 is Running and Ready. -{% endcapture %} -{% include templates/concept.md %} diff --git a/docs/concepts/index.md b/docs/concepts/index.md index b6e6d89a0b320..72d1ecebb20f2 100644 --- a/docs/concepts/index.md +++ b/docs/concepts/index.md @@ -9,7 +9,7 @@ The Concepts section of the Kubernetes documentation is a work in progress. * [Annotations](/docs/concepts/object-metadata/annotations/) #### Controllers -* [StatefulSets](/docs/concepts/controllers/statefulsets/) +* [StatefulSets](/docs/concepts/abstractions/controllers/statefulsets/) ### What's next From 2b35e212cfbd239f0c0a5718ff149bf3bc352295 Mon Sep 17 00:00:00 2001 From: Kenneth Owens Date: Fri, 2 Dec 2016 13:52:31 -0800 Subject: [PATCH 13/13] Moved statefulset concept to abstractions --- .../abstractions/controllers/statefulsets.md | 171 ++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 docs/concepts/abstractions/controllers/statefulsets.md diff --git a/docs/concepts/abstractions/controllers/statefulsets.md b/docs/concepts/abstractions/controllers/statefulsets.md new file mode 100644 index 0000000000000..188284b507abf --- /dev/null +++ b/docs/concepts/abstractions/controllers/statefulsets.md @@ -0,0 +1,171 @@ +--- +assignees: +- bprashanth +- enisoc +- erictune +- foxish +- janetkuo +- kow3ns +- smarterclayton +--- + +{% capture overview %} +**StatefulSets are a beta feature in 1.5. This feature replaces the +PetSets feature from 1.4. Users of PetSets are referred to the 1.5 +[Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/) +for further information on how to upgrade existing PetSets to StatefulSets.** + +A StatefulSet is a Controller that provides a unique identity to its Pods. It provides +guarantees about the ordering of deployment and scaling. +{% endcapture %} + +{% capture body %} + +### When to Use a StatefulSet +StatefulSets are valuable for applications that require one or more of the +following. + +* Stable, unique network identifiers. +* Stable, persistent storage. +* Ordered, graceful deployment and scaling. +* Ordered, graceful deletion and termination. + +In the above, stable is synonymous with persistent across Pod (re) schedulings. +If an application doesn't require any stable identifiers or ordered deployment, +deletion, or scaling, you should deploy your application with a controller that +provides a set of stateless replicas. Such controllers, such as +[Deployment](/docs/user-guide/deployments/) or +[ReplicaSet](/docs/user-guide/replicasets/) may be better suited to your needs. + +### Limitations +* StatefulSet is a beta resource, not available in any Kubernetes release prior to 1.5. +* As with all alpha/beta resources, you can disable StatefulSet through the `--runtime-config` option passed to the apiserver. +* The storage for a given Pod must either be provisioned by a [PersistentVolume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. +* Deleting and/or scaling a StatefulSet down will *not* delete the volumes associated with the StatefulSet. This is done to ensure data safety, which is generally more valuable than an automatic purge of all related StatefulSet resources. +* StatefulSets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. You are responsible for creating this Service. +* Updating an existing StatefulSet is currently a manual process. + +### Components +The example below demonstrates the components of a StatefulSet. + +* A Headless Service, named nginx, is used to control the network domain. +* The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods. +* The volumeClaimTemplates, will provide stable storage using [PersistentVolumes](/docs/user-guide/volumes/) provisioned by a + PersistentVolume Provisioner. + +```yaml +--- +apiVersion: v1 +kind: Service +metadata: + name: nginx + labels: + app: nginx +spec: + ports: + - port: 80 + name: web + clusterIP: None + selector: + app: nginx +--- +apiVersion: apps/v1beta1 +kind: StatefulSet +metadata: + name: web +spec: + serviceName: "nginx" + replicas: 3 + template: + metadata: + labels: + app: nginx + spec: + terminationGracePeriodSeconds: 10 + containers: + - name: nginx + image: gcr.io/google_containers/nginx-slim:0.8 + ports: + - containerPort: 80 + name: web + volumeMounts: + - name: www + mountPath: /usr/share/nginx/html + volumeClaimTemplates: + - metadata: + name: www + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 1Gi +``` + +### Pod Identity +StatefulSet Pods have a unique identity that is comprised of an ordinal, a +stable network identity, and stable storage. The identity sticks to the Pod, +regardless of which node it's (re) scheduled on. + +__Ordinal Index__ + +For a StatefulSet with N replicas, each Pod in the StatefulSet will be +assigned an integer ordinal, in the range [0,N), that is unique over the Set. + +__Stable Network ID__ + +Each Pod in a StatefulSet derives its hostname from the name of the StatefulSet +and the ordinal of the Pod. The pattern for the constructed hostname +is `$(statefulset name)-$(ordinal)`. The example above will create three Pods +named `web-0,web-1,web-2`. +A StatefulSet can use a [Headless Service](/docs/user-guide/services/#headless-services) +to control the domain of its Pods. The domain managed by this Service takes the form: +`$(service name).$(namespace).svc.cluster.local`, where "cluster.local" +is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). +As each Pod is created, it gets a matching DNS subdomain, taking the form: +`$(podname).$(governing service domain)`, where the governing service is defined +by the `serviceName` field on the StatefulSet. + +Here are some examples of choices for Cluster Domain, Service name, +StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods. + +Cluster Domain | Service (ns/name) | StatefulSet (ns/name) | StatefulSet Domain | Pod DNS | Pod Hostname | +-------------- | ----------------- | ----------------- | -------------- | ------- | ------------ | + cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} | + cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} | + kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} | + +Note that Cluster Domain will be set to `cluster.local` unless +[otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). + +__Stable Storage__ + +Kubernetes creates one [PersistentVolumes](/docs/user-guide/volumes/) for each +VolumeClaimTemplate, as specified in the StatefulSet's volumeClaimTemplates field +In the example above, each Pod will receive a single PersistentVolume with a storage +class of `anything` and 1 Gib of provisioned storage. When a Pod is (re) scheduled onto +a node, its `volumeMounts` mount the PersistentVolumes associated with its +PersistentVolume Claims. Note that, the PersistentVolumes associated with the +Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted. +This must be done manually. + +### Deployment and Scaling Guarantee + +* For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}. +* When Pods are being deleted, they are terminated in reverse order, from {N-1..0}. +* Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready. +* Before a Pod is terminated, all of its successors must be completely shutdown. + +When the web example above is created, three Pods will be deployed in the order +web-0, web-1, web-2. web-1 will not be deployed before web-0 is +[Running and Ready](/docs/user-guide/pod-states), and web-2 will not be deployed until +web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before +web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and +becomes Running and Ready. + +If a user were to scale the deployed example by patching the StatefulSet such that +`replicas=1`, web-2 would be terminated first. web-1 would not be terminated until web-2 +is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and +is completely shutdown, but prior to web-1's termination, web-1 would not be terminated +until web-0 is Running and Ready. +{% endcapture %} +{% include templates/concept.md %}