Multiple TLS Client Certs: Only the First Certificate is Being Accepted by Nginx #4234
Comments
How are you doing this exactly? |
Yes but we parse the cert in go from the secret in order to generate the one nginx will use. Not sure this is supported |
|
@johannes-gehrs did you find a way around this problem? I'm having the same problem in NGINX Ingress 0.25.0 with kubernetes version 1.15.2. @aledbf is this not supported when using NGINX as the ingress controller in kubernetes? If not, do you know if there are plans to support it? Thanks for any info!
in case it's still not clear how to dupe this: Steps to dupe:
|
|
It sure looks like this functionality should be supported in nginx-ingress. Even on the kubernetes/ingress-nginx readme documentation it says that it's supported and even has instructions on how to do it (from https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/auth/client-certs):
So it seems like it should be supported, I just must be doing something wrong. |
|
It is still unresolved for us. Using a pure nginx setup from the official Docker containers with a basic config we were unable to replicate the issue. If I were to investigate further I would try to compare the configs and then enable options from the generated ingress config one by one. My suspicion is that one of the options used interferes with nginx's ability to accept multiple certs. I don't think that it is necessarily your fault @slideroi. |
|
Thanks for the reply! |
|
Only the first CA is used, the "rest" of the PEM block is ignored with Decode function expects repeated calls to get the "next" PEM block. Unfortunately a show stopper for us. It's really too bad the documentation is contrary to this. |
|
Please use the image |
|
Thank you for looking at this. Attempting to replace the existing image provides: |
|
@hobti01 please use |
|
Thank you @aledbf for the updated versions but they do not solve the 400 error for us. We continue to receive While a read of the code looks like it would solve the multiple CA issue, that does not seem to be the first and current problem with our configuration. |
|
The explanation/fix seems weird to me because I am pretty sure that the additional certificate did show up in the generated nginx config. But I may misremember or just may have been mistaken back when we tried this. Will surely try out this fix. Thanks @aledbf. |
|
Thanks @aledbf for the fix. Maybe we will pursue our issue in the future but we observed the same issue with "plain" nginx 1.16, so we have not found any nginx based solution. For now we are reverting to a previous architecture that does not use nginx or have the functionality we want in order to pursue other topics. |
@johannes-gehrs please use the test docker image I posted in my previous comment and let me know if the fix works. |
@hobti01 again, if you can open a new issue with something I can use to reproduce the problem, I am more than happy to see what I can do. The hardest part of fixing this kind of issues is always related to the examples to reproduce it. |
|
@aledbf I tested this with |
No. The fix just makes sure we pass all the certs contains in the
Yes. Provide a full example that reproduces the issue. |
|
@aledbf thanks for all your help on this! I tried to reproduce the failure from scratch with all new certs and I haven't been able to. I can reproduce the problem at will with my original two certs, but with any new cert I generate it works just fine. And I can combine either of the original certs with any number of my newly generated certs in any order and it works just fine. There's just something about those original two certs that don't like each other for some reason. I'm pretty sure I used the exact same cert generation settings for them, but I'm not sure how to check that for sure so I'm just going to call it good. Thanks again! Turns out those original two certs that were generated and don't like to live together are problematic because they happened to both be generate with the same subject. |
|
I'm going to include my steps to dupe (which will be steps to verify with the current fix) because I've already got them written up and maybe they'll be useful for someone who is initially trying to test multiple certs in nginx ingress or even just setting it up to start with. Also attaching a zip file with the certs I generated using the instructions below. MultiCertTestFiles.zip 1) Cert generation commands (you can skip these if you're just going to use the cert's I've attached)
2) Install cert and configure PKI
3) Test cert One
4 )Test cert Two
5 )Test both certs combined
|
|
Very welcome, it's good to see everyone get to the bottom of the issues we were seeing. |
|
FYI nginx have closed the issue saying that this is expected behavior.
I guess it makes sense, so the only way forward would be to use different subjects / DNs. |
NGINX Ingress controller version:
0.24.1 but we were also able to replicate the problem in 0.23.0.
Kubernetes version (use
kubectl version):Environment:
uname -a): Linux ip-172-20-81-79 4.9.0-9-amd64 Basic structure #1 SMP Debian 4.9.168-1+deb9u2 (2019-05-13) x86_64 LinuxWhat happened:
When providing multiple client certificates to nginx in one file, i.e. using the secret
auth-tls-secretit did not accept clients authenticating based on the second TLS certificate. It did accept clients authenticating based on the first certificate.When it does not accept the certificate it generates a 400 bad request "The SSL certificate error".
When trying to replicate the issue using a pure nginx config (i.e. not using the ingress) we were not able to replicate the problem.
The generated nginx configuration looks correct to us, i.e. the certificates get put into the nginx config file and this looks fine to us.
What you expected to happen:
According to the nginx docs you can put multiple client certs in the file referenced in
ssl_client_certificateand nginx should accept any one of those. This is also the behavior we saw when trying to isolate the behavior in a pure Docker/Nginx setup, but in ingress nginx it does not work for unknown reasons.How to reproduce it (as minimally and precisely as possible):
Use the TLS client auth feature with more than one certificate.
Anything else we need to know:
Motivation for use of this feature is having zero-downtime certificate exchange, blue-green-style.
Let us know if we can provide additional info to ease replication of this issue.
The text was updated successfully, but these errors were encountered: