Skip to content
This repository has been archived by the owner on Jan 17, 2023. It is now read-only.

Latest commit

 

History

History
263 lines (197 loc) · 11.9 KB

README.md

File metadata and controls

263 lines (197 loc) · 11.9 KB

Episode 144 : Exploring The State of K8s on Windows

  • Hosted by @jayunit100
  • Recording date: 2021-01-28

Table of Contents

  • 00:00:00 - Welcome to TGIK and introductions/announcements
  • 00:15:18 - Looking at how NetworkPolicy Truth tables work on unsupported clusters
  • 00:32:29 - Viewing working NetworkPolicies in Windows with Calico 3.16
  • 00:45:01 - Lookint at CAPI on Windows
  • 00:55:40 - How CNIs are installed with post-kubeadm/preBootstrap commands on Windows
  • 01:02:15 - How runtimeClasses, taints, nodeSelectors work together to schedule Windows pods
  • 01:13:53 - How Cluster API works on NSX for windows with VSphere and NSX as the network plane
  • 01:15:34 - How to use hub.docker.com to lookup windows images matching your OS
  • 01:19:07 - Looking at CSI Proxy, briefly
  • 01:26:45 - The most interesting problem in containerd and windows networking : The CNI ADD Codepath!

News of the Week

VMware Tanzu related stuff

Show Notes

Windows K8s current news

The ecosystem...

CNI providers on Windows

Images

Storage

CAPI and windows

CSI on Windows

Add Windows Monitor to Prom Operator

CSI proxy

image builder

Containerd

Priveliged containers

kubectl exec privileged powershell.exe get-hnsnetwork will be working

CNI

Calico

Antrea

Create Folders

New-Item -ItemType Directory -Force -Path C:\k\antrea
New-Item -ItemType Directory -Force -Path C:\k\antrea\logs
New-Item -ItemType Directory -Force -Path C:\k\antrea\bin
New-Item -ItemType Directory -Force -Path C:\var\log\kube-proxy

Set IP in kubeadm-flags.envs and set Node_name environment variable

$env:HostIP = (
  Get-NetIPConfiguration | Where-Object {
     $_.IPv4DefaultGateway -ne $null -and $_.NetAdapter.Status -ne "Disconnected"
  }
).IPv4Address.IPAddress
$file = 'C:\var\lib\kubelet\kubeadm-flags.env'
$newstr="--node-ip=" + $env:HostIP
$raw = Get-Content -Path $file -TotalCount 1
$raw = $raw -replace ".$"
$new = "$($raw) $($newstr)`""
Set-Content $file $new


[Environment]::SetEnvironmentVariable("NODE_NAME", (hostname).ToLower())

Download Antrea Components

curl.exe -LO "https://raw.githubusercontent.com/vmware-tanzu/antrea/master/hack/windows/Install-OVS.ps1"

Install Open Vswitch

We need to enable test signing if this is the first time...

invoke-expression "bcdedit /set TESTSIGNING ON" 
Restart-computer

./Install-OVS.ps1

Important that installing on the host requires a kubeconfig to get the secrets and nodes for the demo we have given the rights to system:nodes C:/etc/kubernetes/kubelet.conf << this needs to be changed in future!

Setup services

This cannot be done over ssh (RDP or terminal in) ``` powershell cd c:\k\antrea

curl.exe -LO https://raw.githubusercontent.com/vmware-tanzu/antrea/master/hack/windows/Helper.psm1
curl.exe -LO http://w3-dbc302.eng.vmware.com/rcao/image/containerd/antrea-agent.exe

mv antrea-agent.exe c:\k\antrea\bin
Import-Module ./helper.psm1

Install-AntreaAgent -KubernetesVersion "v1.19.1" -KubernetesHome "c:/k" -KubeConfig "C:/etc/kubernetes/kubelet.conf" -AntreaVersion "v0.12.0" -AntreaHome "c:/k/antrea"
New-KubeProxyServiceInterface

nssm install kube-proxy "c:/k/kube-proxy.exe" "--proxy-mode=userspace --kubeconfig=C:/etc/kubernetes/kubelet.conf --log-dir=c:/var/log/kube-proxy --logtostderr=false --alsologtostderr"
nssm install antrea-agent "c:/k/antrea/bin/antrea-agent.exe" "--config=c:/k/antrea/etc/antrea-agent.conf --logtostderr=false --log_dir=c:/k/antrea/logs --alsologtostderr --log_file_max_size=100 --log_file_max_num=4"

nssm set antrea-agent DependOnService kube-proxy ovs-vswitchd
nssm set antrea-agent Start SERVICE_DELAYED_START

start-service kube-proxy
start-service antrea-agent            
```

Windows commands !

Do it all in powershell

  • Tail a file: Get-Content myfile.txt -Wait
  • Find a file: Get-Childitem –Path C:\ -Recurse -Name *ctr*
  • List processes:Get-Process *ovs* | Format-Table -Property Name
  • Chocolatey: Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1')), then run refreshenv in the shell
  • use Vim: choco install vim -y
  • You can run ctr --namespace=k8s.io containers list
  • Look at VNIC Events Get-WinEvent Microsoft-Windows-Hyper-V-VmSwitch-Operational and Microsoft-Windows-Hyper-V-Compute-Operational
  • sort cmd line output:| Sort-Object
  • look at the containers hcsdiag.exe list all
  • look at container networks hnsdiag.exe list all
  • look at all containers ctr.exe --namespace=k8s.io c ls
  • version [System.Environment]::OSVersion.Version
  • grep ~ Select-String, i.e. Get-WinEvent -ListLog * | Out-String -Stream | Select-String SSH

Log File locations

Kube-proxy: c:/var/log/kube-proxy Antrea-Agent: c:/k/antrea/logs Kubelet: c:/var/log/kubelet Containers: c:/var/log/containers OVS: c:/openvswitch/var/log/openvswitch

.

REFERENCES!

kubernetes/kubernetes#98102 antrea-io/antrea#1581 containerd/containerd#4851 https://www.youtube.com/watch?v=FKoVztEQHss

OTHER STUFF

  • cluster api internal stuff

  • What is Windows sig currently focused on?
    • Having a proper end-to-end post merge release blocking test suite using containerd. Ravi.G, James et al. are working on it
    • 1.21 KEP
      • KEP 2258: Use kubectl to view system service logs #2271
      • KEP 1981: Windows privileged container KEP updates for alpha #2288

Reference Links