CNI networking: Antrea

[jbeda]

Monday or Tuesday:

• Pick a topic and create/use an issue in the repo
• If applicable coordinate with an upstream project if they're interested in attending the show
• Create the graphic
• Schedule on Youtube
• Add scheduled episode to TGIK playlist on YT
• Create short link in rebrandly
• Tweet about it from your personal account
  • Post tweets on internal #tgik slack channel and cc @castrojo and Vykky

Before episode:

• Create HackMD page for live notes. Start with README template in this repo
• Paste HackMD URL on internal #tgik channel and cc @castrojo or other assistant
  • Assistant adds ~5-10 news items from around the web as possible "News of the Week"
• Get everything tested well before episode so you can debug
• Use internal slack channel to ensure that AV setup is good

After show (ideally right away but can wait until Monday):

• Create/submit PR for this repo
  • Episode directory and README from hackmd
  • Any other files from the episode that might be useful
  • Update playlist.md with episode
  • Feel free to self merge
• Edit YT description to point to episode directory here
• OBS/OS Upgrade if applicable (Never on the day of a show)

[jayunit100]

Hiya joe, so, for me learning antrea is mostly about comparing it with calico.

• Calico uses BGP, so nodes are routable. Antrea uses OVS. Sot the first most interesting thing is to run route -n on an antrea node, and notice that there are these "gateways", each with a x.y.z.1 IP address:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.17.0.1      0.0.0.0         UG    0      0        0 br-phy
10.10.0.0       0.0.0.0         255.255.255.0   U     0      0        0 antrea-gw0
10.10.1.0       10.10.1.1       255.255.255.0   UG    0      0        0 antrea-gw0
10.10.2.0       10.10.2.1       255.255.255.0   UG    0      0        0 antrea-gw0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-phy

• Now you won't see that in calico ! thats because theres NO SWITCH in calico, and the nodes just broadcast routing information to each other about pods... So the next thing i think to do is spin up prometheus this is REALLY easy , you just git clone antrea, and run kubectl apply -f build/yamls/antrea-prometheus.yml NOTE YOU HAVE TO ENABLE PROMETHEUS METRICS IN THE ANTREA CONFIGMAP IN KUBESYSTEM when doing this. Then of course just expose the master IP to prometheus however you want to. You can then look at the tables for OVS and see what happens when you scale a deployment , to the number of OVS routing tables.

• FINALLY i guess antctl is the next most obvious thing to play with, where you can see network policies. My fave why to see this is run an nginx daemonset/networkpocliy combo, i.e.e

➜  kind git:(master) ✗ cat nginx.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ds
spec:
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx
➜  kind git:(master) ✗ cat pol.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: web-deny-all
spec:
  podSelector:
    matchLabels:
      app: web
  ingress: []

andd then use antctl and

/tmp » kubectl -n kube-system exec -it antrea-agent-2kksz ovs-ofctl dump-flows br-int | grep table=90                 

to see how the new conj rules are created to track the firewalls. This is different from calico bc in calico, the equivalent command woudl be to use iptables-save (and diff before/after policy applied) to see the iptables rules created to implement network policies.

Anyways, full summary is here https://github.com/jayunit100/k8sprototypes/blob/master/2020kubecon/cni_debugging.md and the metrics stuff is here https://github.com/jayunit100/k8sprototypes/tree/master/antrea_metrics !

• One other thing you can do is kill the antrea controller manager which is an apiserver extension, and try to schedule apod, hint - it works. so, that helps to show the difference between antrea, the CNI, from antrea the networkpolicy/controller functionality. its easy for newcomers to cni to get confused that, actually, you can be a cni without a global controller , so would be a good pedagogical mechanism i guess.

anyways have fun !just some ideas

[antoninbas]

Two things that are pretty cool to demo IMO:

• Use Traceflow (packet tracing in the Pod network) along with NetworkPolicies to show traffic being accepted or dropped. antctl query endpoint can be demo'd simultaneously to show policies applied to specific Pods and aggregated NetworkPolicy metrics can be show using vanilla kubectl.

• Enable flow export and show flow information using Kibana dashboards (https://github.com/vmware-tanzu/antrea/blob/master/docs/network-flow-visibility.md). This requires a "beefier" cluster to run the ELK stack, but we provide the manifests.

Happy to provide additional pointers if needed.

[jianjuns]

For Traceflow, check this one: https://github.com/vmware-tanzu/antrea/blob/master/docs/traceflow-guide.md

If we want to go further to dump OVS config, flows, and Antrea internal state, these two docs list some useful commands:
https://github.com/vmware-tanzu/antrea/blob/master/docs/antctl.md
https://github.com/vmware-tanzu/antrea/blob/master/docs/troubleshooting.md

[jayunit100]

        if [[ ! -d antrea ]] ; then
            git clone https://github.com/vmware-tanzu/antrea.git
        fi
        pushd antrea/ci/kind
            ./kind-setup.sh create antrea
        popd

how to run antrea on kind