k3s-kustomize-skaffold-k3d-example

An example local k3s development environment using kustomize, skaffold and k3d.

Table of Contents

• Features
• Prerequisites
• Usage
• Kustomize configuration

Features

• Bootstraps k3s cluster in Docker using k3d
• Skaffold loads docker images directly into the k3s cluster
• Skaffold uses kustomize for building and deploying k8s manifests using local overlay
• An example node.js app will be bootstrapped with File sync and Port forward enabled

Prerequisites

NB. The setup is tested on macOS Monterey.

The first prerequisite is to install go-task in order to make the setup a bit easier:

brew install go-task/tap/go-task

The following prerequisites are used in order to create and manage the local K3s cluster:

• Docker Desktop
• Kubectl
• Kustomize
• Skaffold
• K3d

If you don't have them installed yet you can install them using install-prerequisites task:

task install-prerequisites

Usage

Create k3s cluster:

NB If you want to change the amount of k3s agents argument e.g. k3d:create-cluster -- <number_of_agents>

task k3d:create-cluster

Make sure your KUBECONFIG points to k3s cluster context (if not already):

kubectl get nodes

Start the local development environment:

task skaffold:dev

An example node.js app is available at:

localhost:3000

Make some changes to src/index.js and they will be synchronized to the pod(s) running the app.

Delete the k3s cluster:

task k3d:delete-cluster

Delete images that are built by Skaffold and stored on the local Docker daemon:

task docker:rmi

Kustomize configuration

Kustomize configuration is based on Directory Structure Based Layout in order to use multiple environments with different configuration. In order to use different clusters remember to specify the corresponding context before applying changes using Skaffold.

├── base
│   ├── deployment.yaml
│   ├── hpa.yaml
│   ├── kustomization.yaml
│   └── service.yaml
└── overlays
    ├── local
    │   ├── deployment-patch.yaml
    │   ├── hpa-patch.yaml
    │   ├── kustomization.yaml
    └── test
        ├── deployment-patch.yaml
        ├── hpa-patch.yaml
        └── kustomization.yaml