consul/connect: Enable running multiple ingress gateways per Nomad agent #9849
Conversation
Connect ingress gateway services were being registered into Consul without an explicit deterministic service ID. Consul would generate one automatically, but then Nomad would have no way to register a second gateway on the same agent as it would not supply 'proxy-id' during envoy bootstrap. Set the ServiceID for gateways, and supply 'proxy-id' when doing envoy bootstrap. Fixes #9834
| @@ -516,7 +518,7 @@ func TestTaskRunner_EnvoyBootstrapHook_gateway_ok(t *testing.T) { | |||
|
|
|||
| // the only interesting thing on bootstrap is the presence of the cluster, | |||
| // everything is configured at runtime through xDS | |||
| require.Equal(t, "my-ingress-service", out.Node.Cluster) | |||
| require.Equal(t, "ingress-gateway", out.Node.Cluster) | |||
There was a problem hiding this comment.
Seems to be a quirk of Consul, if you pass -proxy-id is uses ingress-gateway as the envoy cluster name, otherwise the service name
There was a problem hiding this comment.
Do we know if it's a bug or deliberate? Seems like it'd run into collisions too.
There was a problem hiding this comment.
Looked closer, it's because it can namespace on id now, e.g.
"node": {
"cluster": "ingress-gateway",
"id": "_nomad-task-85e65316-c6f9-44ba-3bc1-ea8cd2dfd493-group-ingress-group-my-ingress-service-8080",
"metadata": {
"namespace": "default",
"envoy_version": "1.16.0"
}
},
There was a problem hiding this comment.
Guess I should add that to the test
| @@ -516,7 +518,7 @@ func TestTaskRunner_EnvoyBootstrapHook_gateway_ok(t *testing.T) { | |||
|
|
|||
| // the only interesting thing on bootstrap is the presence of the cluster, | |||
| // everything is configured at runtime through xDS | |||
| require.Equal(t, "my-ingress-service", out.Node.Cluster) | |||
| require.Equal(t, "ingress-gateway", out.Node.Cluster) | |||
There was a problem hiding this comment.
Do we know if it's a bug or deliberate? Seems like it'd run into collisions too.
Co-authored-by: Tim Gross <[email protected]>
|
I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. |
Connect ingress gateway services were being registered into Consul without
an explicit deterministic service ID. Consul would generate one automatically,
but then Nomad would have no way to register a second gateway on the same agent
as it would not supply 'proxy-id' during envoy bootstrap.
Set the ServiceID for gateways, and supply 'proxy-id' when doing envoy bootstrap.
Fixes #9834