VXLAN instead of pcap

[inercia]

In addition to the currently used encapsulation format, Weave could support VXLAN encapsulation. Maybe this would help in some environments...

[rade]

To clarify, is the objective here to gain performance by performing encapsulation - in this case vxlan - in the kernel? Or something else?

[inercia]

I think Weave could initialize a VXLAN device (if not given in command line), add dynamically the MAC addresses of all the peers in the distributed switch, then send/receive packets to/from this device... Otherwise, Weave could use VXLAN just as an encapsulation format, but I don't think this would really improve performance...

[rade]

send/receive packets to/from this device

What's doing that? The containers, i.e. they are wired directly into vxlan? Or the weave router? In which case I don't see the point since we'd still end up doing packet capture/injection, which is where the performance bottleneck is.

[inercia]

You can connect containers directly to a VXLAN, or you could leave the capture/injection work to the Weave router. VXLAN is designed for environments with many, big VLANs where it could help by reducing the number of connections/traffic as it is based in multicast, so the performance gain depends on the scope of Weave...

[inercia]

BTW, Flannel's implementation could be used for getting some tips...

[rade]

Well, we need to be clear what this issue is meant to address. It can't be both. I suspect the cost/benefit ratio of implementing peer connectivity over vxlan is too great; the use cases where this would yield a significant (i.e. order of magnitude) performance benefit - i.e. in very large deployments - will likely be a tiny fraction of our potential user base for a good while.

Using vxlan as an alternative to capture/injection, otoh, would deliver significant performance increases to a much greater spectrum of potential users. It's a vast amount of work though, more so since we want to make it play nicely, and ideally transparently, with current weave (so we can fall back to that when vxlan won't work).

BTW, Flannel's implementation could be used for getting some tips...

Yes, I've seen that.

[inercia]

I think this could be closed once #1438 (fast datapath) is merged, right?