From 91e98b46e947345a79ef90e623c866edd761b34c Mon Sep 17 00:00:00 2001 From: Anirudh Ramanathan Date: Fri, 3 Mar 2017 17:20:57 -0700 Subject: [PATCH] Add a section for prerequisites (#171) * Adding prerequisites * address comments (cherry picked from commit 6556451fb309b77d1a689fdf4a6209504ba16810) --- docs/running-on-kubernetes.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md index 1824fd6d2de98..660bbaca8da90 100644 --- a/docs/running-on-kubernetes.md +++ b/docs/running-on-kubernetes.md @@ -3,9 +3,15 @@ layout: global title: Running Spark on Kubernetes --- -Support for running on [Kubernetes](https://kubernetes.io/) is available in experimental status. The feature set is +Support for running on [Kubernetes](https://kubernetes.io/docs/whatisk8s/) is available in experimental status. The feature set is currently limited and not well-tested. This should not be used in production environments. +## Prerequisites + +* You must have a running Kubernetes cluster with access configured to it using [kubectl](https://kubernetes.io/docs/user-guide/prereqs/). If you do not already have a working Kubernetes cluster, you may setup a test cluster on your local machine using [minikube](https://kubernetes.io/docs/getting-started-guides/minikube/). +* You must have appropriate permissions to create and list [pods](https://kubernetes.io/docs/user-guide/pods/), [nodes](https://kubernetes.io/docs/admin/node/) and [services](https://kubernetes.io/docs/user-guide/services/) in your cluster. You can verify that you can list these resources by running `kubectl get nodes`, `kubectl get pods` and `kubectl get svc` which should give you a list of nodes, pods and services (if any) respectively. +* You must have an extracted spark distribution with Kubernetes support, or build one from [source](https://github.com/apache-spark-on-k8s/spark). + ## Setting Up Docker Images Kubernetes requires users to supply images that can be deployed into containers within pods. The images are built to @@ -81,7 +87,7 @@ the driver container as a [secret volume](https://kubernetes.io/docs/user-guide/ ### Kubernetes Clusters and the authenticated proxy endpoint Spark-submit also supports submission through the -[local kubectl proxy](https://kubernetes.io/docs/user-guide/connecting-to-applications-proxy/). One can use the +[local kubectl proxy](https://kubernetes.io/docs/user-guide/accessing-the-cluster/#using-kubectl-proxy). One can use the authenticating proxy to communicate with the api server directly without passing credentials to spark-submit. The local proxy can be started by running: