Support Configuring Consul Connect Clients

[banks]

Consul Connect manages Mutual TLS identities in an automated fashion.

Servers that wish to accept Connect traffic currently need to integrate directly with Connect's API/SDK to correctly authorize incoming traffic, however for systems that only need to be a client of upstream Connect-enabled services, all they need is the correct certificates.

Supporting Connect certificate fetching via Consul-Template enables some powerful integrations:

• Edge load balancers or API gateways that are currently configured via CT to populate backends can now obtain Connect certs and so maintain end-to-end TLS without needing another sidecar proxy.
• Simple applications that only need to consume Connect-protected services can have certificates managed and delivered via CT without any re-tooling and without needing a sidecar proxy.

To enable this CT needs to be able to load a few new resources from Consul APIs:

Root Certificates

First, CT needs to be able to watch the root certs and render any changed trusted roots to disk in a configurable location that can be passed to application config. Exactly how that's achieved and configured needs some thought, but as an MVP I'd suggest being able to configure both nginx and haproxy correctly to perform mTLS with a backend running on Connect.

Leaf Certificate

CT needs to be able to fetch a leaf certificate for a specific service name. That service name is typiclally the name of the "gateway" or loadbalancer - for example let's call it ingress. In Connect this service identity becomes the source for Intentions (access rules) and any backend setup would need to have an explicit Intention allowing traffic from ingress.

Leaf certificates can only be issued to processes with an appropriate ACL token with service:write for that service name. It's assumed that the additional machinery for Consul ACL tokens is sufficient for this though.

Connect Endpoint Discovery

Finally, CT needs to be able to populate a backend with connect-enabled endpoints. That means that for a given target service (e.g. web) the instance IP:port pairs to populate in the backend definition need to be the sidecar-proxies of those web instances not the actual web instances themselves. This is trivial: a new resource that targets /v1/health/connect/:service instead of the existing one that targets /v1/health/service/:service.

MVP

As mentioned above, with all these in place, the goal would be that we can produce a guide for setting up an existing nginx or HAProxy based loadbalancer to be able to route traffic directly to a Connect enabled backend without another sidecar proxy. If anything is found missing in this issue to enable that workflow we should add that to the scope.

For now we will leave aside any of the newer L7 routing features in Consul 1.6.0 as they are significantly more subtle to integrate into proxies.

[eikenb]

#1304 addresses the technical/code changes and the reference docs. I'm still working on the nginx/haproxy docs and will close this when those are finished.

[eikenb]

Docs done; #1306

[eikenb]

Working on release now.

[cbarbara-okta]

@banks & @eikenb - can you confirm if these new features will work against consul connect 1.5.3? Or is this >= 1.6 only?

[banks]

@cbarbara-okta my understanding is that this should work with any version on Consul > 1.2.0 since it's just exposing the basic Connect endpoints that haven't changed since first introduction.