Fix SSL-Passthrough functionality

[vasrem]

What this PR does / why we need it:
This PR fixes the SSL-Passthrough functionality since whitelisting isn't working correctly.

Which issue this PR fixes :
fixes #2096

Special notes for your reviewer:
We did some tests regarding the functionality after each commit contained in that PR and the results are presented below.

• Current functionality:

Scenario	HTTP	HTTPS with SSL-Offloading	HTTPS with SSL-Passthrough
Proxy Protocol + SSL-Passthrough	❌	✅	❌ *
SSL-Passthrough	✅	❌ **	❌ *
-	✅	✅	✅ ***

Notes
* Whitelisting isn't taken into account so anyone can access the service
** If whitelisting is defined, then it blocks everyone except 127.0.0.1
*** Returns 404 which is correct.

---

• After commit acd5356:

Scenario	HTTP	HTTPS with SSL-Offloading	HTTPS with SSL-Passthrough
Proxy Protocol + SSL-Passthrough	❌	✅	❌ *
SSL-Passthrough	✅	✅	❌ *
-	✅	❌	✅ **

Notes
* Whitelisting isn't taken into account so anyone can access the service
** Returns 404 which is correct.

---

• After commit 5d22d27:

Scenario	HTTP	HTTPS with SSL-Offloading	HTTPS with SSL-Passthrough
Proxy Protocol + SSL-Passthrough	❌	✅	❌ *
SSL-Passthrough	✅	✅	❌ *
-	✅	✅	✅ **

Notes
* Whitelisting isn't taken into account so anyone can access the service
** Returns 404 which is correct.

---

• After commit 1f88c8e:

Scenario	HTTP	HTTPS with SSL-Offloading	HTTPS with SSL-Passthrough
Proxy Protocol + SSL-Passthrough	❌	✅	✅ *
SSL-Passthrough	✅	✅	✅ *
-	✅	✅	✅ **

Notes
* I cannot create a response with HTTP status code 403, so instead I send an empty reply. But in that case whitelisting is working correctly.
** Returns 404 which is correct.

Additional contributors: @nrobert13

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: vasrem
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: bowei

If they are not already assigned, you can assign the PR to them by writing /assign @bowei in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[aledbf]

Closing. We explicitly point out that there is no support for annotations in SSL passthrough https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#ssl-passthrough
This is a great feature but we will not start adding features to the go proxy.

To this (or any other feature) be implemented we need to rollback #614 and fix the issues in one place, NGINX. The reason why I wrote 614 is related to the no preservation of the source IP address

[vasrem]

This implies that the users who use SSL-Passthrough should develop the authentication layer on their apps. In my opinion this generates a lot of work for a feature that will be deprecated as soon as the ingress controller has this capability implemented.

I can understand and I find completely reasonable that you want to refactor this go proxy and take advantage of the NGINX, but since then all the users that are using services which require SSL-Passthrough have no IP-Whitelisting on a layer above their application, which leads to the fact that they just cannot use them in production.

But anyway, do you know when approximately will we have that security feature available?

[sig-piskule]

@aledbf
Can you comment on this thread further?

I have been digging to figure out why whitelisting and ssl-passthrough were not working together and finally came across this thread. I need this feature for exactly the reason @vasrem pointed out. I migrated off of external load balancers on to nginx-ingress controller, which was a big enough migration that simply SSL passthrough was the best option. If Nginx doesn't support whitelisting + ssl_passthrough, then I need to engage developers to add new features and/or rearchitect how the system handles SSL certificates. This is something I was planning on doing, just not on such a short timeframe (you know, technical debt and all).

You referenced this issue #614. I looked at it, but I was not able to determine how, or if this would resolve the whitelisting + SSL Passthrough issue. Additionally, I wasn't sure if the work completed in pull request 614 opened up any work arounds I haven't thought of.

Any comments/tips?