Add new local Gateway sharing same deployment as ingress Gateway.

[JRBANCEL]

Related to #212.

The idea is to create a new Gateway named knative-local-gateway (more consistent with knative-ingress-gateway than cluster-local-gateway). This Gateway binds to the same Envoy Pods as the Istio ingress Gateway, but on a different port (8081 because 8080 is already used by Istio). There is a corresponding Kubernetes Service mapping named knative-local-gateway living in knative-serving (cleaner than cluster-local-gateway living in istio-system) mapping port 80 to port 8081.
knative-local-gateway is added to the default Istio Gateways, this way the VirtualServices of local Knative Services bind to it (i.e. ready to serve traffic), but because the KIngress status uses the first local Gateway in its status, the Services of type ExternalName still point to cluster-local-gateway (and it is not accessible to external requests since 8081 is not exposed through the LoadBalancer Service).

Effectively, this is a no-op, but in the next release we can remove reference to cluster-local-gateway in default config-istio and there will be no traffic interruption and the requests will flow to knative-local-gateway.

After that, everything related to cluster-local-gateway can be safely removed, including the Deployment, which is the end goal here.

/hold
/assign @tcnghia
/assign @ZhiminXiang
Let me know what you think.

[JRBANCEL]

/assign @nak3

[JRBANCEL]

I am looking into the test failure.
/retest

[JRBANCEL]

I need more data.
/retest

[tcnghia]

pull-knative-sandbox-net-istio-latest is a lot flakier than pull-knative-sandbox-integration-test because it is using istio-1.5+istioctl. As long as the flakes aren't new you can ignore them. (Majority of these flakes are in visibility/path tests).

I am root-causing these new flakes (they also affect istio-1.5 + helm - but a little bit less), in the mean time apologies for the noises.

[tcnghia]

@JRBANCEL does the istio-ingressgateway Deployment has this port open by default, or do we need to change the Deployment when setting up Istio to have this port? Thanks

[JRBANCEL]

@JRBANCEL does the istio-ingressgateway Deployment has this port open by default, or do we need to change the Deployment when setting up Istio to have this port? Thanks

The Service selecting the Deployment Pods has port 80 and port 443 open to allow external traffic.
See https://github.com/knative-sandbox/net-istio/blob/master/third_party/istio-1.5.7-helm/istio-ci-no-mesh.yaml#L774.

We specifically don't want that port (8081) to be open, because then cluster local Knative Services would be accessible from outside.

[tcnghia]

Yes, this migration plan sounds good to me.

[ZhiminXiang]

This plan sounds good to me.

Now the envoy pods will have the configurations of both public Ingress and cluster-local Ingress. I think envoy should have the mechanism to isolate the public config and cluster-local config. But we may still want to double check that users cannot access cluster-local ingress from outside of the cluster just in case.

[tcnghia]

You may want to add a RELEASE NOTE section in the PR description to make it easier to remember to include in release notes / announcements.

Otherwise
/lgtm

[JRBANCEL]

But we may still want to double check that users cannot access cluster-local ingress from outside of the cluster just in case.

Yes, I tested that. Since the cluster local Gateway is on a port that is not exposed internally, it can't be accessed.
Either you hit <PUBLIC_IP>:8081 and you can't connect, or you hit <PUBLIC_IP>:80 and it goes to the public Gateway and you get a 404 for a cluster local service.

[ZhiminXiang]

/lgtm

[ZhiminXiang]

/approve

[knative-prow-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JRBANCEL, ZhiminXiang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [JRBANCEL,ZhiminXiang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tcnghia]

@JRBANCEL is this ready for /unhold?

[tcnghia]

Also I assigned you knative/serving#6269 since I thought it would be related to this work.

[JRBANCEL]

/hold cancel

[mattmoor]

This should be using system.Namespace()

[mattmoor]

In fact, we have a helper for synthesizing service hostnames that this should be using too in place of GetClusterDomainName()