Take TLS Certificates by string

[rfdickerson]

Right now the library assumes a file that contains the certificate. It would be helpful to have the library also take a String that's Base64 encoded, as well. Bluemix users will receive a Base64 string for their certificate through the Bluemix environment variables to be used to connect with MongoDB, for instance.

[billabt]

What is the format of the String you want to pass? PEM? PKCS12? If PEM, do you also have the key in String form? If PKCS12, I assume you'll also have the password, correct? Trying to figure out if this is going to be a Linux only feature or available on macOS as well. Any assistance here is greatly appreciated...

[tfrank64]

@billabt I believe we get a PEM from Bluemix. Everything we get is in one base64 string which I can send to you directly.

[billabt]

BlueSSLService v0.12.13 contains the new configuration API and the underlying support. However, it's only been tested minimally. Please let me know if you have any problems with it and open a new issue. If you do run into problems, it would be helpful if you can supply a standalone test case and attach it to the new issue. Thanks.

[rfdickerson]

Thanks, @billabt ! We will check it out and see if it works for our MongoKitten branch. Regarding the cross platform issue- is there any way to get a pk12 out of a string that's in PEM format all by using Swift and your tools?

[billabt]

I'm still looking at the best way to implement on macOS. I assumed you needed it first on Linux so that's why I released that implementation as soon as it was done rather than wait for the macOS version. I think it'll be possible but I've some more research to do...

[billabt]

Actually since it's not really done until we get or don't get the macOS version, I'm going to re-open the issue. So if there are problems, just use this issue... Thanks.

[tfrank64]

I'm unsure of why, but I'm getting a seg fault in SSLService.swift here.
LLDB gives me this

Process 374 stopped
* thread #1: tid = 374, 0x00007ffff77fe877 libcrypto.so.1.0.0`CRYPTO_add_lock + 71, name = 'TodoList', stop reason = signal SIGSEGV: invalid address (fault address: 0x65d3)
    frame #0: 0x00007ffff77fe877 libcrypto.so.1.0.0`CRYPTO_add_lock + 71
libcrypto.so.1.0.0`CRYPTO_add_lock:
->  0x7ffff77fe877 <+71>: addl   (%r12), %r13d
    0x7ffff77fe87b <+75>: movl   0xc(%rsp), %r8d
    0x7ffff77fe880 <+80>: movq   %rbp, %rdx
    0x7ffff77fe883 <+83>: movl   %ebx, %esi

I'm leveraging the new init method here.

[billabt]

I need more info... Backtrace? Standalone (outside Bluemix) reproducible scenario?

[tfrank64]

Here is the backtrace I believe:

* thread #1: tid = 117, 0x00007ffff77fe877 libcrypto.so.1.0.0`CRYPTO_add_lock + 71, name = 'TodoList', stop reason = signal SIGSEGV: invalid address (fault address: 0x65d3)
  * frame #0: 0x00007ffff77fe877 libcrypto.so.1.0.0`CRYPTO_add_lock + 71
    frame #1: 0x00007ffff7868574 libcrypto.so.1.0.0`RSA_up_ref + 36
    frame #2: 0x00007ffff7bb5b64 libssl.so.1.0.0`___lldb_unnamed_symbol269$$libssl.so.1.0.0 + 196
    frame #3: 0x00007ffff7bb4cb4 libssl.so.1.0.0`SSL_new + 196
    frame #4: 0x00000000004a5a4c TodoList`SSLService.prepareConnection(socket=Socket.Socket @ 0x00007fffffff84a8, self=<unavailable>, $error=<unavailable>) throws -> UnsafeMutablePointer<ssl_st> + 588 at SSLService.swift:937
    frame #5: 0x000000000049f317 TodoList`SSLService.onConnect(socket=Socket.Socket @ 0x00007fffffff8618, self=<unavailable>, $error=<unavailable>) throws -> () + 215 at SSLService.swift:445
    frame #6: 0x00000000004a8622 TodoList`protocol witness for SSLServiceDelegate.onConnect(socket : Socket) throws -> () in conformance SSLService + 66 at SSLService.swift:0
    frame #7: 0x0000000000468018 TodoList`Socket.connect(host=<unavailable>, port=<unavailable>, self=Socket.Socket @ 0x00007fffffff9ab8, $error=<unavailable>) throws -> () + 11272 at Socket.swift:1833
    frame #8: 0x000000000071faa8 TodoList`static Socket.open(hostname=<unavailable>, port=<unavailable>, options=<unavailable>, self=<unavailable>, $error=<unavailable>) throws -> MongoTCP + 7912 at SSL.swift:53
    frame #9: 0x00000000006fa73f TodoList`Server.makeConnection(host=<unavailable>, authenticatedFor=<unavailable>, self=<unavailable>, $error=<unavailable>) throws -> Server.Connection + 2271 at Server.swift:454
    frame #10: 0x00000000006f6c3f TodoList`Server.init(clientSettings=<unavailable>, $error=<unavailable>) throws -> Server + 5855 at Server.swift:229
    frame #11: 0x00000000006f992a TodoList`Server.__allocating_init($error=<unavailable>) throws -> Server + 1338 at Server.swift:0
    frame #12: 0x000000000042bdde TodoList`TodoList.init(dbConfiguration=<unavailable>) -> TodoList + 9998 at TodoList.swift:71
    frame #13: 0x000000000042c6e1 TodoList`TodoList.__allocating_init(DatabaseConfiguration) -> TodoList + 561 at TodoList.swift:0
    frame #14: 0x0000000000422023 TodoList`main + 2931 at main.swift:83
    frame #15: 0x00007ffff52c0f45 libc.so.6`__libc_start_main + 245
    frame #16: 0x000000000041dae9 TodoList`_start + 41

To reproduce, clone my branch of TodoList-MongoDB.
Initialize the DatabaseConfiguration with something like this in main.swift, using your own certificate and mongoDB uri:

        let cert = "<base64_encoded_cert>"
        let uri = "mongodb://<username>:<password>@bluemix-sandbox-dal-9-portal.5.dblayer.com:19889,bluemix-sandbox-dal-9-portal.4.dblayer.com:19889/admin?ssl=true"
        let service = Service(name: "", label: "", plan: "", tags: [""], credentials: ["uri": uri, "ca_certificate_base64": cert])
        databaseConfiguration = DatabaseConfiguration(with: service)

Then run on Linux and you should get a segmentation fault. I can send you my mongoDB credentials if needed.

[billabt]

I tried doing what you suggested and I'm unable to get it working. Can you narrow it down to simpler test case that illustrates the problem? It'd be extremely helpful. Otherwise, because of my other project commitments outside Swift@IBM (plus still on the mend from a bout with pneumonia), I'm not sure when I'll be able to get to it...

[billabt]

In reviewing the code, I found a potential issue with the handling of the passed string. I'm not sure if it'll fix your problem but it's probably worth trying. v0.12.15.

[tfrank64]

I still seem to have issues with v0.12.15. I will try to get a simple example together.

[billabt]

I was able to reproduce the problem and determine the cause. The last set of changes before this, changed the way closed sockets were being handled. This caused a problem where the SSLService was being deinitialized too early. The latest BlueSocket and BlueSSLService correct this problem.

[tfrank64]

@billabt Everything appears to be working on Linux, including Bluemix deployment. Thanks a lot.
The next thing we need is for SSL support on macOS. Not as crucial, but still important.

[billabt]

Support for SSL is available on macOS using a PKCS12. If you're talking about doing it via the same mechanism as on Linux using a PEM formatted string, I don't think we're gonna be able to do it. There doesn't appear (at least that I can find) a way of configuring Secure Transport using a PEM formatted string but I'm still looking.

[rfdickerson]

Ideally, of course, we would want to be able to handle the PEM that Bluemix gives us and get it working on Mac. But maybe this isn't a deal breaker- since, probably macOS users will be using a development environment, and can probably do the PEM to PKCS12 conversion on the command line.

The steps taken could be:

1. Run cf env to get the certificate Base64 PEM file
2. Convert the base64 to a file cert.pem
3. Use openssl command line tools to convert the pem to cert.pkcs12.
4. Store the pkcs12 contents to a config.json file
5. Use swift-configuration to load the config.json file contents

[billabt]

Yes, that's definitely a possibility. But, before you go down that road (unless it's time critical), give me a few more days to explore being able to use the PEM string...

[tfrank64]

@billabt Just wanted to check in to see if you have had time to explore options of using the PEM string on macOS? Let us know if we can help.

[billabt]

@tfrank64 Haven't had the chance. Been working on BlueRSA and a BlueSSLService issue... It's still in my queue.

[benaubin]

@billabt Any updates? I can't figure out a way to do it via Apple's Certificate Key and Trust Services.

[billabt]

So far, neither can I... 😝

[benaubin]

@billabt Would it be possible to do with OpenSSL?

[benaubin]

So, to answer my own question: It is possible with OpenSSL - it's just a mess. Here's my solution (feel free to use it however you'd like - I'm releasing the code and instructions into the public domain).

1. You have to define an Objective C file to create a STACK_OF(X509) certificates. This is my first time ever writing something in ObjC, but this is what I have:

#import <Foundation/Foundation.h>
#import <openssl/safestack.h>
#import <openssl/x509.h>
#import <openssl/objects.h>

@interface X509_STACK_CREATOR:NSObject
- (STACK_OF(X509))createStack:(NSArray*)certificates;
@end

@implementation X509_STACK_CREATOR
- (STACK_OF(X509)) createStack:(NSArray*)certificates{
    STACK_OF(X509) *stack = NULL;
    stack = sk_X509_new_null();
    
    for(int i = 0; i < [certificates count]; i++){
        X509 *certificate = (__bridge X509 *)([certificates objectAtIndex:i]);
        sk_X509_push(stack, certificate);
    }
    
    return *stack;
}
@end

2. Add the @interface to your swift bridging header.
3. Use BIO_new BIO_puts and PEM_read_bio_X509 to create the X509 certificates.
4. Convert the private key to an RSA private key.

    func toEVP(privateKey: SecKey) throws -> UnsafeMutablePointer<EVP_PKEY>? {
        var error: Unmanaged<CFError>?
        guard let privateKeyData = SecKeyCopyExternalRepresentation(privateKey, &error) as NSData? else {
            throw error!.takeRetainedValue() as Error
        }
        
        var rsaPrivateKey: UnsafeMutablePointer<RSA>?
        var privateKeyBytes: UnsafePointer<UInt8>? = privateKeyData.bytes.assumingMemoryBound(to: UInt8.self)
        d2i_RSAPrivateKey(&rsaPrivateKey, &privateKeyBytes, privateKeyData.length)
        
        var evpPrivateKey = EVP_PKEY_new()
        EVP_PKEY_set1_RSA(evpPrivateKey, rsaPrivateKey)
        
        return evpPrivateKey
    }

5. PKCS12_create to make a certificate - generate a random passphrase and keep it in memory.
6. Use BIO_read to read the certificate into memory - then store the encrypted certificate in NSTemporaryDirectory() - use deinit to remove it later.
7. When ready to use, just pass in the PKCS12 file and passphrase.

Sadly, it seems to give a bad access error during the PKCS12 creation. I probably messed up with the ObjC code.

[billabt]

I've done this before in another application and inside BlueSSLService when running on Linux. However, to incorporate it into BlueSSLService, it would require linking to both the macOS libraries and OpenSSL when running on macOS. This is an unnecessary burden (the OpenSSL) for most apps. Probably better to do it on a case by case basis with an extension to BlueSSLService.

[billabt]

Due to lack of support in the macOS Security module, this will unfortunately fall into the category of won't fix. However, in those cases where it's absolutely necessary and you don't mind the bloat, you can use the method described by @benaubin above. If someone has another suggestion, I'd be more than happen to try it.