Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFI: ca_cert_file #506

Closed
huggsboson opened this issue Feb 23, 2017 · 5 comments
Closed

RFI: ca_cert_file #506

huggsboson opened this issue Feb 23, 2017 · 5 comments

Comments

@huggsboson
Copy link

huggsboson commented Feb 23, 2017 •
edited
Loading

Hi,

I've got envoy setup in a kubernetes cluster and I was looking to add Mutual Auth TLS to an ingress listener. Box has an interesting TLS setup where the files we typically get provided are:

  1. serviceName.crt: Just the cert for the service. (Not the chain)
  2. serviceName.key: The private key
  3. serviceName.pem: CACerts.. certs for Intermediate CA's (we run an intermediate CA for each service type) for services that this service trusts.
  4. serviceName.combined.pem: The whole cert chain for the service.

So on the ingress listener I've added:

ssl_context: {
    cert_chain_file:  "/box/etc/pki/%s.crt" % ingressServiceName,
    private_key_file: "/box/etc/pki/%s.key" % ingressServiceName,
    ca_cert_file:     "/box/etc/pki/%s.pem" % ingressServiceName,
},

Now if I comment out the ca_cert_file line I am able to curl the endpoint and get a response.
If I leave the line in I'm unable to establish a TLS connection. Please note in all of the below. I've got the ca file setup to trust the service itself and I'm connecting to the service with its own cert, own key, and the ca file which trusts itself.

We've used this type of cert setup for a number of different programs and not had an issue before. I'm curious if there's something special needed for envoy.

Note the curl command line I used was (it successfully verifies the cert with it's cacert file):

$ curl -v --cacert serviceName.pem --cert serviceName.crt --key serviceName.key https://localhost:11000
curl -v --cacert /box/etc/pki/hello-www-envoy.pem --cert /box/etc/pki/hello-www-envoy.crt --key /box/etc/pki/hello-www-envoy.key https://localhost:11000
* About to connect() to localhost port 11000 (#0)
*   Trying ::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 11000 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /box/etc/pki/hello-www-envoy.pem
  CApath: none
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: [email protected],CN=localhost,OU=Security Operations,O="Box, Inc.",L=Los Altos,ST=California,C=US
* 	start date: Feb 23 05:55:08 2017 GMT
* 	expire date: Feb 23 05:55:08 2019 GMT
* 	common name: localhost
* 	issuer: [email protected],CN=Development Testing Authority,OU=Security Operations,O="Box, Inc.",L=Los Altos,ST=California,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: localhost:11000
> Accept: */*
...

If I openssl x509 -in serviceName.crt -text -noout, here are some snippets:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 84062 (0x1485e)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=California, L=Los Altos, O=Box, Inc., OU=Security Operations, CN=Development Testing Authority/[email protected]
        Validity
            Not Before: Feb 23 05:55:08 2017 GMT
            Not After : Feb 23 05:55:08 2019 GMT
        Subject: C=US, ST=California, L=Los Altos, O=Box, Inc., OU=Security Operations, CN=localhost/[email protected]
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:f1:7d:4b:7a:97:48:29:67:a2:05:26:73:b5:3d:
                    ed:61:e5:25:04:87:8c:d9:15:74:64:fa:15:2b:b8:
                    56:62:df:40:f7:f6:8e:61:21:6b:65:d0:62:dc:cb:
                    a4:05:12:59:bd:58:64:a6:1a:3c:e7:9e:86:6c:fb:
                    51:30:02:e9:22:37:a7:57:46:f1:1e:f3:92:ea:fc:
                    fc:c2:98:2a:b1:76:27:14:3c:ae:ea:f7:03:35:6f:
                    f9:2d:c5:68:c1:6c:2d:70:43:98:67:78:18:24:9f:
                    8c:8d:bd:16:dc:93:71:57:18:79:f3:95:cc:48:12:
                    09:9e:54:8c:c1:eb:32:69:5f:98:21:47:0e:da:4f:
                    34:cb:11:91:71:00:d2:71:6b:96:46:83:0a:20:98:
                    90:62:25:45:6c:df:a0:5f:47:88:72:bd:66:f1:1e:
                    8c:84:cc:ae:14:07:b5:8c:ed:ed:a4:96:d9:e2:08:
                    5f:5c:b1:6a:b5:21:e5:71:44:70:e8:8c:61:89:74:
                    6a:31:9b:64:b2:ed:e3:2d:28:65:5e:8d:b2:a8:0b:
                    44:0c:6e:36:5a:8f:89:f2:76:f9:6d:a6:78:b2:05:
                    1e:56:13:64:12:39:7d:87:2c:87:a6:1e:2b:1a:fc:
                    5d:2f:f7:f1:17:73:0a:62:b6:6f:94:82:b5:82:8d:
                    7e:85
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Client Authentication, TLS Web Server Authentication
            X509v3 Subject Key Identifier:
                26:89:E0:3A:81:70:C1:BE:75:77:D7:F7:32:34:96:C6:42:4F:6F:28
            X509v3 Authority Key Identifier:
                keyid:51:11:F9:9F:30:C4:AC:12:F3:38:6C:E7:63:A4:02:44:D8:81:D2:5E

            X509v3 Subject Alternative Name:
                DNS:localhost
            X509v3 CRL Distribution Points:

                Full Name:
                  URI:http://cert.dev.box.net:4780/dev_test.crl

    Signature Algorithm: sha256WithRSAEncryption
         90:79:26:00:da:a6:ce:7a:61:2c:f0:8a:23:94:08:de:be:62:
         c1:51:e3:87:9c:64:7d:b0:f7:ae:c6:64:74:e6:d1:6c:a7:d2:
         86:1e:3d:26:69:e4:b5:26:66:a4:1f:bf:2d:d0:c8:75:72:c7:
         a6:fd:eb:92:42:05:01:6d:96:a0:63:4f:49:82:d8:2e:d0:30:
         9e:e8:d9:20:0b:53:f1:47:30:02:9b:53:2f:d5:85:3c:fd:78:
         c9:91:ab:c4:97:51:4a:8f:d9:e3:f9:41:1b:2a:05:69:0e:83:
         7a:5e:eb:a9:3c:73:a4:50:68:63:7b:7e:d8:f3:d6:06:fd:70:
         ea:b6:c4:49:bd:3f:b3:cc:dd:bf:81:1a:78:fd:6f:a0:2c:d4:
         93:80:35:d5:58:c5:b0:e7:84:90:49:86:90:ff:9b:b5:1b:27:
         7c:05:65:94:86:e6:08:d2:8b:5f:f6:38:47:07:a6:21:71:d1:
         7f:ca:97:1a:2f:5a:b7:81:3d:57:00:06:91:eb:ea:11:4f:a1:
         a1:d7:b1:ad:52:f1:74:eb:ac:02:a5:ec:51:66:2a:0c:d6:6d:
         18:dc:ef:61:77:9c:67:20:cf:f7:1b:29:cd:ff:02:90:23:e9:
         78:2f:91:a5:9b:b8:03:87:12:e1:59:09:fa:d8:75:90:5e:2a:
         8c:44:20:19

If I do the same on the serviceName.pem:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 19 (0x13)
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: C=US, ST=California, L=Los Altos, O=Box, Inc., OU=Security Operations, CN=Development Intermediate Authority/[email protected]
        Validity
            Not Before: Mar 13 18:19:01 2014 GMT
            Not After : Mar  8 18:19:01 2034 GMT
        Subject: C=US, ST=California, L=Los Altos, O=Box, Inc., OU=Security Operations, CN=Development Testing Authority/[email protected]
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:a6:5f:85:34:dd:6d:97:b5:99:d8:cd:3e:f9:04:
                    67:f2:ed:a4:5f:89:9b:ec:e9:e0:5a:26:c0:87:cc:
                    f4:3a:75:7b:1f:bb:7c:67:09:58:7f:49:f2:c9:df:
                    c0:4e:b9:77:b5:75:70:59:69:b5:00:22:9c:84:bc:
                    48:9f:b4:6e:0c:83:6e:3e:50:4c:05:10:c4:0b:65:
                    c0:06:37:47:85:6c:d5:11:41:6a:80:72:d0:dc:eb:
                    5b:ab:c4:88:fe:08:97:0f:71:6f:e2:cf:99:9d:ea:
                    f5:08:62:98:a9:bc:e7:05:7e:d0:47:1e:bb:a9:ea:
                    87:7e:16:e8:ea:5e:cb:cd:77:81:62:6e:33:89:69:
                    be:8a:a6:30:e6:95:0d:26:ec:2d:79:6e:a4:72:cb:
                    30:65:64:df:84:c5:89:95:49:99:fa:73:55:3c:b7:
                    0d:d9:0b:f6:73:20:89:22:58:98:04:a0:4a:a3:a0:
                    e5:00:65:45:38:32:fe:f6:52:fe:99:e9:08:c4:1d:
                    64:a7:49:00:7c:1a:68:8f:0a:2d:be:f5:03:53:d4:
                    ba:5b:af:18:bd:40:a0:d0:23:09:e1:59:09:ff:a1:
                    c6:73:06:0c:eb:80:8f:2a:79:de:e5:a0:7b:2f:e6:
                    19:c8:80:da:96:3e:01:d0:24:eb:c4:aa:04:b1:70:
                    3e:8f
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Key Usage:
                Certificate Sign, CRL Sign
            X509v3 Subject Key Identifier:
                51:11:F9:9F:30:C4:AC:12:F3:38:6C:E7:63:A4:02:44:D8:81:D2:5E
            X509v3 Authority Key Identifier:
                keyid:F5:13:7E:C6:4B:B3:7E:15:78:A3:56:FE:10:B7:E1:98:DC:AA:47:31

    Signature Algorithm: sha1WithRSAEncryption
         36:1b:73:cb:db:be:d8:c3:37:8a:a2:c2:97:af:0d:21:bc:9d:
         56:d8:55:0e:ea:f1:ae:d8:8d:49:7d:c5:02:12:14:50:91:81:
         40:71:f8:4d:b3:89:88:9d:32:b6:66:e4:c1:1c:87:ca:46:e1:
         43:1a:b8:8e:d0:9b:41:d4:1a:b6:47:68:5e:0a:5a:80:9d:10:
         63:de:2e:1f:87:11:9e:b2:fa:2a:2a:25:40:06:5d:52:54:0a:
         f0:26:7b:c3:29:0e:6e:1f:e9:5e:c7:ee:c5:43:02:02:81:72:
         d3:76:50:a7:c9:a6:f3:f2:b8:90:bb:f4:6d:d9:0d:43:f0:82:
         49:9e:4a:73:f0:45:a1:0a:07:4a:c4:6f:b0:e0:f7:d5:9f:5c:
         9b:93:33:3f:d0:15:40:f8:34:29:15:5b:5e:f5:c8:bf:66:ad:
         6f:26:3a:50:c2:c7:59:82:b8:65:60:7d:a4:d0:d2:13:04:72:
         7f:ca:a6:96:60:9e:e8:98:75:d8:fa:a7:18:23:fa:5c:0d:48:
         e9:0e:71:6a:71:1a:ae:54:e2:7a:b3:cf:2b:3f:aa:91:6f:98:
         68:df:2f:20:b6:f0:30:05:ff:da:f3:7a:6b:da:fc:63:b6:07:
         f9:0f:a2:29:b7:28:0e:c9:87:1e:4c:14:78:80:70:56:20:b2:
         64:a9:db:ac

If I connect using openssl s_client -CAfile serviceName.pem -key serviceName.pem -cert serviceName.crt -showcerts -connect localhost:11000:

CONNECTED(00000003)
depth=1 C = US, ST = California, L = Los Altos, O = "Box, Inc.", OU = Security Operations, CN = Development Testing Authority, emailAddress = [email protected]
verify error:num=2:unable to get issuer certificate
issuer= C = US, ST = California, L = Los Altos, O = "Box, Inc.", OU = Security Operations, CN = Development Intermediate Authority, emailAddress = [email protected]
verify return:0
140380564461384:error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca:s3_pkt.c:1259:SSL alert number 48
140380564461384:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:184:
---
Certificate chain
 0 s:/C=US/ST=California/L=Los Altos/O=Box, Inc./OU=Security Operations/CN=localhost/[email protected]
   i:/C=US/ST=California/L=Los Altos/O=Box, Inc./OU=Security Operations/CN=Development Testing Authority/[email protected]
-----BEGIN CERTIFICATE-----
MIIEoDCCA4igAwIBAgIDAUheMA0GCSqGSIb3DQEBCwUAMIGxMQswCQYDVQQGEwJV
UzETMBEGA1UECAwKQ2FsaWZvcm5pYTESMBAGA1UEBwwJTG9zIEFsdG9zMRIwEAYD
VQQKDAlCb3gsIEluYy4xHDAaBgNVBAsME1NlY3VyaXR5IE9wZXJhdGlvbnMxJjAk
BgNVBAMMHURldmVsb3BtZW50IFRlc3RpbmcgQXV0aG9yaXR5MR8wHQYJKoZIhvcN
AQkBFhBzZWN1cml0eUBib3guY29tMB4XDTE3MDIyMzA1NTUwOFoXDTE5MDIyMzA1
NTUwOFowgZ0xCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRIwEAYD
VQQHEwlMb3MgQWx0b3MxEjAQBgNVBAoTCUJveCwgSW5jLjEcMBoGA1UECxMTU2Vj
dXJpdHkgT3BlcmF0aW9uczESMBAGA1UEAxMJbG9jYWxob3N0MR8wHQYJKoZIhvcN
AQkBFhBzZWN1cml0eUBib3guY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEA8X1LepdIKWeiBSZztT3tYeUlBIeM2RV0ZPoVK7hWYt9A9/aOYSFrZdBi
3MukBRJZvVhkpho8556GbPtRMALpIjenV0bxHvOS6vz8wpgqsXYnFDyu6vcDNW/5
LcVowWwtcEOYZ3gYJJ+Mjb0W3JNxVxh585XMSBIJnlSMwesyaV+YIUcO2k80yxGR
cQDScWuWRoMKIJiQYiVFbN+gX0eIcr1m8R6MhMyuFAe1jO3tpJbZ4ghfXLFqtSHl
cURw6IxhiXRqMZtksu3jLShlXo2yqAtEDG42Wo+J8nb5baZ4sgUeVhNkEjl9hyyH
ph4rGvxdL/fxF3MKYrZvlIK1go1+hQIDAQABo4HSMIHPMAwGA1UdEwEB/wQCMAAw
DgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATAd
BgNVHQ4EFgQUJongOoFwwb51d9f3MjSWxkJPbygwHwYDVR0jBBgwFoAUURH5nzDE
rBLzOGznY6QCRNiB0l4wFAYDVR0RBA0wC4IJbG9jYWxob3N0MDoGA1UdHwQzMDEw
L6AtoCuGKWh0dHA6Ly9jZXJ0LmRldi5ib3gubmV0OjQ3ODAvZGV2X3Rlc3QuY3Js
MA0GCSqGSIb3DQEBCwUAA4IBAQCQeSYA2qbOemEs8IojlAjevmLBUeOHnGR9sPeu
xmR05tFsp9KGHj0maeS1JmakH78t0Mh1csem/euSQgUBbZagY09Jgtgu0DCe6Nkg
C1PxRzACm1Mv1YU8/XjJkavEl1FKj9nj+UEbKgVpDoN6XuupPHOkUGhje37Y89YG
/XDqtsRJvT+zzN2/gRp4/W+gLNSTgDXVWMWw54SQSYaQ/5u1Gyd8BWWUhuYI0otf
9jhHB6YhcdF/ypcaL1q3gT1XAAaR6+oRT6Gh17GtUvF066wCpexRZioM1m0Y3O9h
d5xnIM/3GynN/wKQI+l4L5Glm7gDhxLhWQn62HWQXiqMRCAZ
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=California/L=Los Altos/O=Box, Inc./OU=Security Operations/CN=localhost/[email protected]
issuer=/C=US/ST=California/L=Los Altos/O=Box, Inc./OU=Security Operations/CN=Development Testing Authority/[email protected]
---
Acceptable client certificate CA names
/C=US/ST=California/L=Los Altos/O=Box, Inc./OU=Security Operations/CN=Development Testing Authority/[email protected]
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 1837 bytes and written 2700 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID:
    Session-ID-ctx:
    Master-Key: 42244C4894BD87C38512330FC05C0E3233A925A58F2B4174D13FF10106670BAA13FE9C30BE6928D7927C6FF079A79B33
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1487868695
    Timeout   : 300 (sec)
    Verify return code: 2 (unable to get issuer certificate)
---

Also:

bash-4.1# openssl version
OpenSSL 1.0.1e-fips 11 Feb 2013
@huggsboson
Copy link
Author

Oh also:

# envoy --version

envoy  version: 5e9f81/RELEASE

@huggsboson
Copy link
Author

Here are the messages when I pass -msg to openssl (with all the bytes removed):

CONNECTED(00000003)
>>> TLS 1.2 Handshake [length 00f2], ClientHello
<<< TLS 1.2 Handshake [length 003b], ServerHello
<<< TLS 1.2 Handshake [length 04ae], Certificate
depth=2 C = US, ST = California, L = Los Altos, O = "Box, Inc.", OU = Security Operations, CN = Development Intermediate Authority, emailAddress = [email protected]
verify error:num=2:unable to get issuer certificate
issuer= C = US, ST = California, L = Los Altos, O = "Box, Inc.", OU = Security Operations, CN = Development Root Authority, emailAddress = [email protected]
verify return:0
<<< TLS 1.2 Handshake [length 014d], ServerKeyExchange
<<< TLS 1.2 Handshake [length 00d3], CertificateRequest
<<< TLS 1.2 Handshake [length 0004], ServerHelloDone
>>> TLS 1.2 Handshake [length 0d41], Certificate
>>> TLS 1.2 Handshake [length 0046], ClientKeyExchange
>>> TLS 1.2 Handshake [length 0108], CertificateVerify
>>> TLS 1.2 ChangeCipherSpec [length 0001]
>>> TLS 1.2 Handshake [length 0010], Finished
<<< TLS 1.2 Alert [length 0002], fatal unknown_ca

@huggsboson
Copy link
Author

huggsboson commented Feb 23, 2017 •
edited
Loading

It looks like neither the client nor the server can verify the issuer up to the root authority.

@huggsboson
Copy link
Author

When I tracked down and added the very very root CA to the CAFile it seems to be working working a little bit better from the client's perspective. I'm assuming if I mirror the same thing on the server side it will get better.

@huggsboson
Copy link
Author

Just to add more color here. openssl needs to be able to trace back to a self-signed rooted certificate. I did not have those in my pem file. NSS (curl) and java are less finicky about it.

wolfguoliang pushed a commit to wolfguoliang/envoy that referenced this issue Jan 23, 2021
@kkfinkkfin
zh-translation: /configuration/http/http_filters/grpc_stats_filter.rst (
87b557b 
envoyproxy#506)

* update docs/root/configuration/http/http_filters/grpc_stats_filter.rst

* update
jpsim pushed a commit that referenced this issue Nov 28, 2022
@junr03 @jpsim
android: wire up on trailers (#514)
cefea43 
Description: this PR wires up on trailers callback for android, which was previously missing.
Risk Level: low - using already battled tested code paths for accumulating headers.
Testing: pending testing with gRPC on the Lyft android app

Fixes #506

Signed-off-by: Jose Nino <[email protected]>
Signed-off-by: JP Simard <[email protected]>
jpsim pushed a commit that referenced this issue Nov 29, 2022
@junr03 @jpsim
android: wire up on trailers (#514)
0b57e34 
Description: this PR wires up on trailers callback for android, which was previously missing.
Risk Level: low - using already battled tested code paths for accumulating headers.
Testing: pending testing with gRPC on the Lyft android app

Fixes #506

Signed-off-by: Jose Nino <[email protected]>
Signed-off-by: JP Simard <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@huggsboson