Skip to content

Commit

Permalink
Add docs for pod integration (#1103)
Browse files Browse the repository at this point in the history
* Add examples for pod integration

* Add "Run A Plain Pod" documentation page

* Update run_plain_pods.md according to PR comments

* Minor updates to the run_plain_pods.md doc page
  • Loading branch information
achernevskii authored Sep 29, 2023
1 parent 01e5a1a commit ab8924c
Show file tree
Hide file tree
Showing 3 changed files with 109 additions and 0 deletions.
1 change: 1 addition & 0 deletions site/content/en/docs/tasks/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ As a batch user, you can learn how to:
Kueue supports MPIJob v2beta1, PyTorchJob, TFJob, XGBoostJob, and PaddleJob.
- [Run a Kueue managed KubeRay RayJob](/docs/tasks/run_rayjobs).
- [Submit Kueue jobs from Python](/docs/tasks/run_python_jobs).
- [Run a Kueue managed plain Pod](/docs/tasks/run_plain_pods)

### Platform developer

Expand Down
90 changes: 90 additions & 0 deletions site/content/en/docs/tasks/run_plain_pods.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
---
title: "Run A Plain Pod"
date: 2023-09-27
weight: 6
description: >
Run a Kueue scheduled Pod.
---

This page shows how to leverage Kueue's scheduling and resource management capabilities when running plain Pods.

This guide is for [batch users](/docs/tasks#batch-user) that have a basic understanding of Kueue. For more information, see [Kueue's overview](/docs/overview).

## Before you begin

1. By default, the integration for `v1/pod` is not enabled.
Learn how to [install Kueue with a custom manager configuration](/docs/installation/#install-a-custom-configured-released-version)
and enable the `pod` integration.

A configuration for Kueue with enabled pod integration would look like follows:
```yaml
apiVersion: config.kueue.x-k8s.io/v1beta1
kind: Configuration
integrations:
frameworks:
- "pod"
podOptions:
# You can change namespaceSelector to define in which
# namespaces kueue will manage the pods.
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: NotIn
values: [ kube-system, kueue-system ]
# Kueue uses podSelector to manage pods with particular
# labels. The default podSelector will match all the pods.
podSelector:
matchExpressions:
- key: kueue-job
operator: In
values: [ "true", "True", "yes" ]
```
2. Pods that belong to other API resources managed by Kueue are excluded from being queued by `pod` integration.
For example, pods managed by `batch/v1.Job` won't be managed by `pod` integration.

3. Kueue will inject a `kueue.x-k8s.io/managed=true` label to indicate which pods are managed by it.

4. Check [Administer cluster quotas](/docs/tasks/administer_cluster_quotas) for details on the initial Kueue setup.

## Pod definition

When running Pods on Kueue, take into consideration the following aspects:

### a. Queue selection

The target [local queue](/docs/concepts/local_queue) should be specified in the `metadata.labels` section of the Pod configuration.

```yaml
metadata:
labels:
kueue.x-k8s.io/queue-name: user-queue
```

### b. Configure the resource needs

The resource needs of the workload can be configured in the `spec.containers`.

```yaml
- resources:
requests:
cpu: 3
```

### c. Limitations

- A Kueue managed Pod cannot be created in `kube-system` or `kueue-system` namespaces.
- In case of [preemption](/docs/concepts/cluster_queue/#preemption), the Pod will
be terminated and deleted.

## Example Pod

Here is a sample Pod that just sleeps for a few seconds:

{{< include "examples/pods-kueue/kueue-pod.yaml" "yaml" >}}

You can create the Pod using the following command:
```sh
# Create the pod
kubectl apply -f kueue-pod.yaml
```
18 changes: 18 additions & 0 deletions site/static/examples/pods-kueue/kueue-pod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
apiVersion: v1
kind: Pod
metadata:
generateName: kueue-sleep-
labels:
kueue.x-k8s.io/queue-name: user-queue
spec:
containers:
- name: sleep
image: busybox
command:
- sleep
args:
- 3s
resources:
requests:
cpu: 3
restartPolicy: OnFailure

0 comments on commit ab8924c

Please sign in to comment.