SSL Handshake failure with certain websites

[vmeyet]

When visiting certain websites (https://api.hearthstonejson.com) with tls enabled there is an handshake failure which should NOT be the case, both regular browser (Chrome/Firefox) and the ruby lib HTTParty connects perfectly to that website.

Errno 14077410 which corresponds to The Destination Site Does Not Like the Cipher

I believe HTTParty use the same OpenSSL version than Crystal (which is the system one v1.0.2)

Crystal compiler version:

➜   crystal -v
Crystal 0.24.1 (2018-01-27)

LLVM: 5.0.1
Default target: x86_64-apple-macosx

Example of failing code

require "http/client"
uri = URI.parse("https://api.hearthstonejson.com")
response = HTTP::Client.new(uri).get("/")

Stacktrace

SSL_connect: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure (OpenSSL::SSL::Error)
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/openssl/ssl/socket.cr:34:9 in 'initialize'
  from OpenSSL::SSL::Socket::Client::new:context:sync_close:hostname<TCPSocket, OpenSSL::SSL::Context::Client, Bool, String>:OpenSSL::SSL::Socket::Client
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/http/client.cr:657:5 in 'socket'
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/http/client.cr:500:5 in 'exec_internal_single'
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/http/client.cr:486:5 in 'exec_internal'
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/http/client.cr:482:5 in 'exec'
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/http/client.cr:585:5 in 'exec'
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/http/client.cr:329:3 in 'get'
  from src/hsync.cr:25:1 in '__crystal_main'
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/crystal/main.cr:11:3 in '_crystal_main'
  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/crystal/main.cr:112:5 in 'main_user_code'
  from /usr/local/  from /usr/local/Cellar/crystal-lang/0.24.1_2/src/crystal/main.cr:101:7 in 'main'

Sidenote

I don't believe this is a duplicate of #5266 / #3477. Not the same error code, and connecting to https://maps.googleapis.com works perfectly fine for me

I've performed an sslscan to see the supported cypher and got the following ones:

➜   sslscan api.hearthstonejson.com

...
Preferred TLSv1.2  128 bits  ECDHE-ECDSA-AES128-GCM-SHA256 Curve P-256 DHE 256
Accepted  TLSv1.2  128 bits  ECDHE-ECDSA-AES128-SHA        Curve P-256 DHE 256
Accepted  TLSv1.2  128 bits  ECDHE-ECDSA-AES128-SHA256     Curve P-256 DHE 256
Accepted  TLSv1.2  256 bits  ECDHE-ECDSA-AES256-GCM-SHA384 Curve P-256 DHE 256
Accepted  TLSv1.2  256 bits  ECDHE-ECDSA-AES256-SHA        Curve P-256 DHE 256
Accepted  TLSv1.2  256 bits  ECDHE-ECDSA-AES256-SHA384     Curve P-256 DHE 256
Preferred TLSv1.1  128 bits  ECDHE-ECDSA-AES128-SHA        Curve P-256 DHE 256
Accepted  TLSv1.1  256 bits  ECDHE-ECDSA-AES256-SHA        Curve P-256 DHE 256
Preferred TLSv1.0  128 bits  ECDHE-ECDSA-AES128-SHA        Curve P-256 DHE 256
Accepted  TLSv1.0  256 bits  ECDHE-ECDSA-AES256-SHA        Curve P-256 DHE 256
...

which seems to be among the ones that are supposedly supported (https://github.com/crystal-lang/crystal/blob/v0.24.1/src/openssl/ssl/context.cr#L13-L54)

[luislavena]

Hello @vmeyet, thank you for you report.

Since Crystal needs to link with OpenSSL when compiling your script, will be great to know against which version it is linking.

A simple test will be run the following command:

$ crystal eval 'require "openssl"; pp LibSSL::OPENSSL_110, LibSSL::OPENSSL_102'

And confirm the output. For reference, here is the output of my Ubuntu 16.04 installation:

LibSSL::OPENSSL_110 # => false
LibSSL::OPENSSL_102 # => true

The ciphers you linked above refers to ciphers used by the server context, but not the ones used by client context (ie. HTTP::Client).

Client should work against any cipher proposed by server, unless specified in your own context.

As example, here is a non-HTTP client request:

require "socket"
require "openssl"

# nslookup api.hearthstonejson.com
socket = TCPSocket.new("104.27.153.213", 443)
context = OpenSSL::SSL::Context::Client.new

ssl_socket = OpenSSL::SSL::Socket::Client.new(socket, context)

And will notice the same failure as your report (using OpenSSL 1.0.2):

SSL_connect: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure (OpenSSL::SSL::Error)
  from src/openssl/ssl/socket.cr:34:9 in 'initialize'
  from src/openssl/ssl/socket.cr:3:5 in 'new'
  from 3.cr:8:1 in '__crystal_main'
  from src/crystal/main.cr:11:3 in '_crystal_main'
  from src/crystal/main.cr:112:5 in 'main_user_code'
  from src/crystal/main.cr:101:7 in 'main'
  from src/crystal/main.cr:141:3 in 'main'
  from __libc_start_main
  from _start
  from ???

And even a direct openssl s_client connection fails:

$ openssl s_client -connect api.hearthstonejson.com:443
CONNECTED(00000003)
140599705142936:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:769:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 305 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1518794100
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

Also fails when forcing TLS version and preferred cipher:

$ openssl s_client -tls1_2 -cipher ECDHE-ECDSA-AES128-GCM-SHA256 -connect api.hearthstonejson.com:443
CONNECTED(00000003)
140323884238488:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number:s3_pkt.c:362:

So I think the issue might be OpenSSL related than Crystal.

Can you perform these debug methods on your end?

Thank you.

[vmeyet]

Thanks for your answer, launching the code you gave me outputs:

➜  crystal eval 'require "openssl"; pp LibSSL::OPENSSL_110, LibSSL::OPENSSL_102'
LibSSL::OPENSSL_110 # => false
LibSSL::OPENSSL_102 # => false

by checking my openssl version, I got

➜  openssl version
OpenSSL 1.0.2n  7 Dec 2017
➜ /usr/bin/openssl version
OpenSSL 0.9.8zh 14 Jan 2016

It seems that the system version is 0.9.8zh even though my version is 1.0.2n (as per the .zshrc $PATH resolution). It might be due to this, Crystal is probably using the obsolete version (from /usr/bin).

I have to do a bit of nasty stuff to symlink the right version in /usr/bin directory. I'll try to see if it resolves this.

Also the ruby counterpart is using the right version and that's why it worked:

➜  irb
irb(main):001:0> require 'openssl'
=> true
irb(main):002:0> OpenSSL::OPENSSL_LIBRARY_VERSION
=> "OpenSSL 1.0.2n  7 Dec 2017"

[luislavena]

If you're using OSX, you might want to use Homebrew's openssl instead:

https://github.com/Homebrew/brew/blob/master/docs/How-to-build-software-outside-Homebrew-with-Homebrew-keg-only-dependencies.md

Look at the pkg-config option.

Hope that helps.

[vmeyet]

the pkg-config option worked perfectly!

I've added export PKG_CONFIG_PATH=$(brew --prefix)/opt/openssl/lib/pkgconfig to my ~/.zshrc and re-run my script and it worked.

Thanks for you help, I believe this can be closed.

Sorry for the hassle due to my (not great) setup.