Modify loadbalancer docs for new external source ip preservation anno…

…tation

docs/user-guide/load-balancer.md

@@ -7,19 +7,28 @@
 ## Overview
 
 When creating a service, you have the option of automatically creating a
-network load balancer. This provides an
+cloud network load balancer. This provides an
 externally-accessible IP address that sends traffic to the correct port on your
-cluster nodes.
+cluster nodes _provided your cluster runs in a supported environment and is configured with the correct cloud load balancer provider package_.
+
+## External Load Balancer Providers
+
+It is important to note that the datapath for this functionality is provided by a load balancer external to the Kubernetes cluster.
+
+When the service type is set to `LoadBalancer`, Kubernetes provides functionality equivalent to type=`ClusterIP` to pods within the cluster and extends it by programming the (external to Kubernetes) load balancer with entries for the Kubernetes VMs. The Kubernetes service controller automates the creation of the external load balancer, health checks (if needed), firewall rules (if needed) and retrieves the external IP allocated by the cloud provider and populates it in the service object.
 
 ## Configuration file
 
 To create an external load balancer, add the following line to your
 [service configuration file](/docs/user-guide/services/operations/#service-configuration-file):
 
+```json
     "type": "LoadBalancer"
+```
 
 Your configuration file might look like:
 
+```json
     {
       "kind": "Service",
       "apiVersion": "v1",
@@ -37,14 +46,17 @@ Your configuration file might look like:
         "type": "LoadBalancer"
       }
     }
+```
 
 ## Using kubectl
 
 You can alternatively create the service with the `kubectl expose` command and
 its `--type=LoadBalancer` flag:
 
+```bash
     $ kubectl expose rc example --port=8765 --target-port=9376 \
         --name=example-service --type=LoadBalancer
+```
 
 This command creates a new service using the same selectors as the referenced
 resource (in the case of the example above, a replication controller named
@@ -58,6 +70,7 @@ For more information, including optional flags, refer to the
 You can find the IP address created for your service by getting the service
 information through `kubectl`:
 
+```bash
     $ kubectl describe services example-service
     Name:  example-service
     Selector:   app=example
@@ -69,5 +82,70 @@ information through `kubectl`:
     Endpoints:    10.64.0.4:80,10.64.1.5:80,10.64.2.4:80
     Session Affinity: None
     No events.
+```
 
 The IP address is listed next to `LoadBalancer Ingress`.
+
+## Loss of client source IP for external traffic
+
+Due to the implementation of this feature, the source IP for sessions as seen in the target container will *not be the original source IP* of the client. This is the default behavior as of Kubernetes v1.4. However, starting in v1.4, an optional alpha feature has been added
+that will preserve the client Source IP for GCE/GKE environments. This feature will be phased in for other cloud providers in subsequent releases.
+
+## Annotation to modify the LoadBalancer behavior for preservation of Source IP
+In 1.4, an Alpha feature has been added that changes the behavior of the external LoadBalancer feature.
+
+This feature can be activated by adding the alpha annotation below to the metadata section of the Service Configuration file.
+
+```json
+    {
+      "kind": "Service",
+      "apiVersion": "v1",
+      "metadata": {
+        "name": "example-service",
+        "annotations": {
+            "service.alpha.kubernetes.io/external-traffic": "OnlyLocal"
+        }
+      },
+      "spec": {
+        "ports": [{
+          "port": 8765,
+          "targetPort": 9376
+        }],
+        "selector": {
+          "app": "example"
+        },
+        "type": "LoadBalancer"
+      }
+    }
+```
+
+### Alpha Feature Gate for the 'service.alpha.kubernetes.io/external-traffic' annotation
+
+Alpha features are not enabled by default, they must be enabled using the release gate command line flags
+for kube-controller-manager and kube-proxy.
+See [https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/runtimeconfig.md](Runtime feature flags proposal) for more details on feature gate flags.
+
+If this feature is not enabled in your cluster, this annotation in your service configuration will be rejected.
+
+### Implementation across different cloudproviders/environments
+
+Note that this feature is not currently implemented for all cloudproviders/environments.
+This feature does not work for nodePorts yet, so environments/cloud providers with proxy-style load-balancers cannot use it yet.
+
+### Caveats and Limitations when preserving source IPs
+
+GCE/AWS load balancers do not provide weights for their target pools. This was not an issue with the old LB
+kube-proxy rules which would correctly balance across all endpoints.
+
+With the new functionality, the external traffic will not be equally load balanced across pods, but rather
+equally balanced at the node level (because GCE/AWS and other external LB implementations do not have the ability
+for specifying the weight per node, they balance equally across all target nodes, disregarding the number of
+pods on each node).
+
+We can, however, state that for NumServicePods << NumNodes or NumServicePods >> NumNodes, a fairly close-to-equal
+distribution will be seen, even without weights.
+
+Once the external load balancers provide weights, this functionality can be added to the LB programming path.
+*Future Work: No support for weights is provided for the 1.4 release, but may be added at a future date*
+
+Internal pod to pod traffic should behave similar to ClusterIP services, with equal probability across all pods.