Certificates created are not accepted by curl/firefox

[Moschn]

Description

• Type: Bug
• Priority: Major

---

Bug

OS
linux

mbed TLS build:
Version: ab0a804
OS version: Ubuntu 16.04
Configuration: not modified. Directly from the development branch

Peer device TLS stack and version
Curl/Firefox

Expected behavior
Create a signed certificate from a given CA certificate (rootCA.key), CA key (rootCA.pem), key (test.key) and signing request (request.csr). Start a testserver with the new certificate. Connect to the testserver with various clients with the rootCA added to the trusted CAs

See that the certificate is accepted.

Actual behavior
Certificate is not accepted.

The certificate gets accepted by both curl and firefox if instead of using mbedtls, openssl is used to create the certificate.
Steps to reproduce

1. create rootCA.key
   openssl genrsa -out rootCA.key 2048
2. create rootCA.pem
   openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem
3. create test.key
   openssl genrsa -out test.key 2048
4. create request.csr
   openssl req -new -key test.key -out request.csr
5. create certificate:
   • openssl: openssl x509 -req -in request.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out cert_openssl.pem -days 500 -sha256
   • mbedtls: ./cert_write request_file=request.csr issuer_crt=rootCA.pem issuer_key=rootCA.key output_file=cert_mbedtls.pem not_before=20170727125153 not_after=20181209125153
6. simple webserver:
   • openssl: openssl s_server -accept 4433 -cert cert_mbedtls.pem -key test.key -www
   • openssl: openssl s_server -accept 4433 -cert cert_openssl.pem -key test.key -www
   • mbedtls: ./ssl_server2 crt_file=cert_mbedtls.pem key_file=test.key
   • mbedtls: ./ssl_server2 crt_file=cert_openssl.pem key_file=test.key
7. Try to connect to the hosted servers.
   E.g. curl -X GET "https://_hostname_:4433" --cacert rootCA.pem

Further information

It does not matter if a certificate is created using a signing request or not. All certificates that are created with cert_write are not accepted (even though the rootCA is supplied to the clients).

It is also interesting to note that openssl s_client accepts both certificates.

[hanno-becker]

Hi @Moschn,

thanks for raising this! Could you provide your curl version and the error you're receiving?

I tried to reproduce the problem following your steps and noticed the following:

1. Using curl version 7.47.0, I get a server certificate verification failed. error.
2. Using curl version 7.55.0-DEV, things work fine.

Looking into the differences between the OpenSSL and mbed TLS generated certificates, the crucial point seems to be the choice of string-encoding: OpenSSL uses UTF8String for most of the subject and issuer names, while mbed TLS uses PrintableString. Looking at https://tools.ietf.org/html/rfc5280#section-4.1.2.4, this seems acceptable, but we'll check this in detail. For testing, I changed the X509 code to also use UTF8String, and then the certificate gets accepted by curl 7.47.0, too.

Regards,
Hanno

[Moschn]

Hi @hanno-arm

I am using curl v7.47.0 and I get the same error message: curl: (60) server certificate verification failed.

For testing, I changed the X509 code to also use UTF8String, and then the certificate gets accepted by curl 7.47.0, too.

This is great news. Can you check if your changes also apply to firefox?

I also tested with chromium but it is not accepting the certificates because chromium stopped accepting Common Name as the hostname. The DNS/IP entries in the certificates should be in the SubjectAltNames section. Support for this is on the way in pull request #731. The behaviour of chromium can be changed (stackoverflow) and then chromium behaves like firefox. It does not accept the certificate from mbedtls but accepts the certificate from openssl.

[ciarmcom]

ARM Internal Ref: IOTSSL-1591

[hanno-becker]

Hi @Moschn,

This is great news. Can you check if your changes also apply to firefox?

Yes, switching from PrintableString to UTF8String also fixes the issue on my version of Firefox (54.0).

I'm surprised by this behavior of Firefox and curl, as I interpreted RFC5280 as not mandating any particular choice of string format.

Regards,
Hanno

[Moschn]

I checked which ssl library curl is built against on my machine. It turns out curl is built against gnutls. So I tried to connect directly with gnutls (version 3.4.10) after adding the rootCA.pem to the root-ca-store:
gnutls-cli _hostname_ -p 4433.

cert_mbedtls.pem:

- Status: The certificate is NOT trusted. The certificate issuer is unknown. 
*** PKI verification of server certificate failed...
*** Fatal error: Error in the certificate.
*** Handshake has failed
GnuTLS error: Error in the certificate.

cert_openssl.pem: works flawlessly.

Firefox and chromium use NSS as their ssl library.

It looks like gnutls and NSS do not accept PrintableString encoding for certificates even though the specification seems to allow it.

[hanno-becker]

Hi @Moschn,

thanks for checking that! The following might be the problem:

GnuTLS seems to store the trusted certificates in a hash-table, the index of a trusted certificate being the hash of the DER-encoded distinguished name it contains, modulo the fixed hash-index size (127). When GnuTLS verifies a certificate, it only considers those trusted certificates as a potential signer whose DN's have the same hash-index as the issuer DN in the certificate to verify. See https://github.com/gnutls/gnutls/blob/gnutls_3_5_x/lib/x509/verify-high.c#L1281 and https://github.com/gnutls/gnutls/blob/gnutls_3_5_x/lib/x509/verify-high.c#L1297.

Now when mbed TLS's cert_write signs a certificate, it prints the issuer name in its preferred format using PrintableString's, while the original CA certificate rootCA.pem generated by OpenSSL uses UTF8String. Therefore the DER encodings of the issuer field in the signed certificate cert_mbedtls.pem and the subject field in the CA certificate rootCA.pem differ and GnuTLS doesn't consider rootCA.pem as a potential signer for cert_mbedtls.pem.

I manually changed the string types from UTF8String to PrintableString in rootCA.pem and with that change, the mbed TLS certificate got accepted.

Looking at https://tools.ietf.org/html/rfc5280#section-7.1, it seems that checking the DN's DER-encodings for raw equality is not valid and that instead some normalization on the strings should be performed - but this as well as my guess above will need more careful study.

In any case, to increase compatibility and avoid such kinds of problems, one could make the certificate signing routine always use the same string format as the signing certificate.

Regards,
Hanno

[RonEld]

I believe that #468 is related

[hanno-becker]

A small update: While the analysis above still seems to apply to the case where all fields are using the character set of PrintableString, the 'downcast' performed by Mbed TLS is not valid in general, as it reinterprets an arbitrarily encoded string as a PrintableString, regardless of whether it can be represented as such or not. This should be fixed.

[thomas-dee]

Just a side note: I stumbled across the exact same issue and patched mbed TLS accordingly.

[thomas-dee]

Improved bugfix to handle all string types correctly

[thomas-dee]

Since the given patch works great for me. Are there any plans on merging this into the main development branch?

[thomas-dee]

Besides String type issues there is another reason, Firefox does not accept created (and signed) certificates.

char issuer_name[256];
mbedtls_x509write_cert crt;
mbedtls_x509_crt issuer_crt;
...
mbedtls_x509_crt_init( &issuer_crt );
mbedtls_x509write_crt_init( &crt );
...
mbedtls_x509_crt_parse(&issuer_crt, issuer_crt_data, issuer_crt_data_len);
mbedtls_x509_dn_gets( issuer_name, sizeof(issuer_name), &issuer_crt.subject );
...
mbedtls_x509write_crt_set_issuer_name(&crt, issuer_name);
...

The then created PEM certificate will lead to an Unknown CA error in Firefox.

The reason is that mbedtls_x509write_crt_pem writes the data stored in crt backwards. So, the issuer_name components are also written backwards. Since the subject name components in the original issuer certificates are in correct (and not backwards) order, Firefox refuses to accept the given issuer name in the signed certificate.

I'm not sure if this is a Firefox or an Mbed TLS bug.

[hanno-becker]

Hi @thomas-dee,

I looked at the relevant functions in #2005, and while I think there is inconsistency in the way the ASN.1 - LinkedList - String conversions sometimes reverse the order and sometimes don't, it seems that when going from ASN.1 to String and vice versa - as your code does, too - the order should be kept; in other words, I currently cannot unify my understanding of the code with the failure you're observing, nor reproduce the latter.

Could you perhaps provide a minimal reproducible example?

Best,
Hanno

[hanno-becker]

The original problem has been fixed through #1641. The issue regarding ordering of DN attributes is treated separately in #2005.