Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SS concept #1719

Merged
merged 13 commits into from
Dec 2, 2016
4 changes: 4 additions & 0 deletions _data/concepts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,7 @@ toc:
section:
- title: Annotations
path: /docs/concepts/object-metadata/annotations/
- title: Controllers
section:
- title: StatefulSets
path: /docs/concepts/controllers/statefulsets/
171 changes: 171 additions & 0 deletions docs/concepts/controllers/statefulsets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
---
Copy link
Contributor

@foxish foxish Nov 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add assignees here, so we know when this doc has issues raised against it:

---
assignees:
- bprashanth
- enisoc
- erictune
- foxish
- janetkuo
- kow3ns
- smarterclayton

---

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the list. We should have the same list of assignees in all StatefulSet docs

assignees:
- bprashanth
- enisoc
- erictune
- foxish
- janetkuo
- kow3ns
- smarterclayton
---

{% capture overview %}
**StatefulSets are a beta feature in 1.5. This feature replaces the
Pet Sets feature from 1.4. Users of Pet Sets are referred to the 1.5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address the user directly.

"If you've been using PetSet in your cluster in version 1.4, see the Upgrade Guide for further information on how to upgrade existing Pet Sets to StatefulSets."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PetSet as one word?

[Upgrade Guide](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets/)
for further information on how to upgrade existing Pet Sets to StatefulSets.**

A StatefulSet is a Controller that provides a unique identity to its Pods. It provides
guarantees about the ordering of deployment and scaling.
{% endcapture %}

{% capture body %}

### When to Use a StatefulSet
StatefulSets are valuable for applications that require one or more of the
following.

* Stable, unique network identifiers.
* Stable, persistent storage.
* Ordered, graceful deployment and scaling.
* Ordered, graceful deletion and termination.

In the above, by stable, we mean persistent across Pod (re) schedulings.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid "we" constructs. Rephrase:

"Both the unique network identifiers and the persistent storage associated with a StatefulSet are "stable" in the sense that they are persistent across Pod scheduling or re-scheduling."

As it is generally easier to manage, if an application doesn't require any of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"If an application doesn't require any stable identifiers or ordered deployment, deletion, or scaling, you should deploy your application with a controller that provides a set of stateless replicas. Such controllers, such as [Deployment] or [ReplicaSet], are generally easier to manage and may be better suited to your needs."

the above guarantees, and if it is feasible to do so, it should be deployed as
a set of stateless replicas. Before deciding to use a StatefulSet, you should
be certain that a [Deployment](/docs/user-guide/deployments/) or a
[ReplicaSet](/docs/user-guide/replicasets/) is not better suited to your needs.

### Limitations
* StatefulSet is a beta resource, not available in any Kubernetes release prior to 1.5.
* As with all alpha/beta resources, it can be disabled through the `--runtime-config` option passed to the apiserver.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid pronouns. "It" -> "StatefulSet".

Avoid passive voice. "It can be disabled" -> "You can disable StatefulSet through the --runtime-config option that you pass to the Kubernetes apiserver."

* The storage for a given Pod must either be provisioned by a [PersistentVolume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin.
* Deleting and/or scaling a StatefulSet down will *not* delete the volumes associated with the StatefulSet. This is done to ensure safety first, your data is more valuable than an auto purge of all related StatefulSet resources.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"This is done to ensure data safety, which is generally more valuable than an automatic purge of all related StatefulSet resources."

* StatefulSets currently require a [Headless Service](/docs/user-guide/services/#headless-services) to be responsible for the network identity of the Pods. The user is responsible for this Service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address the user directly. "The user is responsible for this Service" -> "You must provide this service for your StatefulSet."

* Updating an existing StatefulSet is currently a manual process.

### Components
The example below demonstrates the components of a StatefulSet.

* A Headless Service, named nginx, is used to control the network domain.
* The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods.
* The volumeClaimTemplates, will provide stable storage using [PersistentVolumes](/docs/user-guide/volumes/) provisioned by a
PersistentVolume Provisioner.

```yaml
Copy link
Member

@janetkuo janetkuo Nov 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use something like {% include code.html language="yaml" file="petset.yaml" ghlink="/docs/user-guide/petset.yaml" %} instead. We may need to rename the file and/or move the file here also

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The annotations example includes its YAML inline. I did the same so that the concepts pages are uniform in how they present structural examples. I'm in favor of using inline for concepts because the include produces a copy to clipboard button, and I don't think we want users copying and pasting from the concepts. If the consensus is that we want to use includes, then we should update the annotations concept for consistency.

---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: "nginx"
replicas: 3
template:
metadata:
labels:
app: nginx
spec:
terminationGracePeriodSeconds: 10
containers:
- name: nginx
image: gcr.io/google_containers/nginx-slim:0.8
ports:
- containerPort: 80
name: web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: www
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
```

### Pod Identity
StatefulSet Pods have a unique identity that is comprised of an ordinal, a
stable network identity, and stable storage. The identity sticks to the Pod,
regardless of which node it's (re) scheduled on.

__Ordinal Index__
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a heading?


For a StatefulSet with N replicas, each Pod in the StatefulSet will be
assigned an integer ordinal, in the range [0,N), that is unique over the Set.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing parenthesis should be a square bracket.


__Stable Network ID__

The hostname of a Pod in a StatefulSet is derived from the name of the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Each Pod in a StatefulSet derives its hostname from the name of the StatefulSet and the ordinal of the Pod."

StatefulSet and the ordinal of the Pod. The pattern for the constructed hostname
is `$(statefulset name)-$(ordinal)`. The example above will create three Pods
named `web-0,web-1,web-2`.
A StatefulSet can use a [Headless Service](/docs/user-guide/services/#headless-services)
to control the domain of its Pods. The domain managed by this Service takes the form:
`$(service name).$(namespace).svc.cluster.local`, where "cluster.local"
is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it).
As each Pod is created, it gets a matching DNS subdomain, taking the form:
`$(podname).$(governing service domain)`, where the governing service is defined
by the `serviceName` field on the StatefulSet.

Here are some examples of choices for Cluster Domain, Service name,
StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods.

Cluster Domain | Service (ns/name) | StatefulSet (ns/name) | StatefulSet Domain | Pod DNS | Pod Hostname |
-------------- | ----------------- | ----------------- | -------------- | ------- | ------------ |
cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} |
cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} |
kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} |

Note that Cluster Domain will be set to `cluster.local` unless
[otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it).

__Stable Storage__

[PersistentVolumes](/docs/user-guide/volumes/), one for each Volume Claim Template,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Kubernetes creates one [PersistentVolume] for each Volume Claim Template, as specified in the StatefulSet's volumeClaimTemplates field."

are created based on the `volumeClaimTemplates` field of the StatefulSet. In the
example above, each Pod will receive a single PersistentVolume with a storage
class of `anything` and 1 Gib of provisioned storage. When a Pod is (re) scheduled onto
a node, its `volumeMounts` mount the PersistentVolumes associated with its
PersistentVolume Claims. Note that, the PersistentVolumes associated with the
Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted.
This must be done manually.

### Deployment and Scaling Guarantee

* For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}.
* When Pods are being deleted, they are terminated in reverse order, from {N-1..0}.
* Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready.
* Before a Pod is terminated, all of its successors must be completely shutdown.

When the web example above is created, three Pods will be deployed in the order
web-0, web-1, web-2. web-1 will not be deployed before web-0 is
[Running and Ready](/docs/user-guide/pod-states), and web-2 will not be deployed until
web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before
web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and
becomes Running and Ready.

If a user were to scale the deployed example by patching the StatefulSet such that
`replicas=1`, web-2 would be terminated first. web-1 would not be terminated until web-2
is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and
is completely shutdown, but prior to web-1's termination, web-1 would not be terminated
until web-0 is Running and Ready.
{% endcapture %}
{% include templates/concept.md %}
7 changes: 6 additions & 1 deletion docs/concepts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,12 @@ The Concepts section of the Kubernetes documentation is a work in progress.

#### Object Metadata

[Annotations](/docs/concepts/object-metadata/annotations/)

* [Annotations](/docs/concepts/object-metadata/annotations/)

#### Controllers
* [StatefulSets](/docs/concepts/controllers/statefulsets/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an "abstractions" directory inside concepts that I'd like "controllers" to be nested under.



### What's next

Expand Down