SSL connect error

[doktorbro]

I try to proof a Jekyll website from this repo.

HTML::Proofer.new("./site/_site", {
  :verbose => true,
  :ssl_verifypeer => false
}).run

If I run the test on Travis, all HTTPS links return a 200.

The same test on my local machine (Ubuntu 14.10) shows me errors for the WHATWG links. GitHub and Travis HTTPS links work as expected.

- ./site/_site/index.html
  *  External link https://html.spec.whatwg.org/ failed: 0 SSL connect error
  *  External link https://html.spec.whatwg.org/multipage/syntax.html failed: 0 SSL connect error
  *  External link https://html.spec.whatwg.org/multipage/semantics.html failed: 0 SSL connect error
  *  External link https://html.spec.whatwg.org/multipage/semantics.html failed: 0 SSL connect error
  *  External link https://html.spec.whatwg.org/multipage/semantics.html failed: 0 SSL connect error
  *  External link https://html.spec.whatwg.org/multipage/tables.html failed: 0 SSL connect error

What is wrong?

[doktorbro]

Proofer 2.0.5 show me the same errors. If I pass the Typhoeus namespace nothing changes.

    HTML::Proofer.new("./site/_site", {
      :verbose => true,
      :typhoeus => {
        :ssl_verifypeer => false
      }
    }).run

I tried a wrong namespace too:

    HTML::Proofer.new("./site/_site", {
      :verbose => true,
      :wrong => {
        :ssl_verifypeer => false
      }
    }).run

Same result as above. Looks like the Typhoeus options are ignored.

[gjtorikian]

I do not think the Typhoeus options are being ignored:

html-proofer/spec/html/proofer/links_spec.rb

Lines 73 to 83 in 4ead382

	it 'should follow redirects' do
	linkWithRedirectFilepath = "#{FIXTURES_DIR}/links/linkWithRedirect.html"
	proofer = run_proofer(linkWithRedirectFilepath)
	expect(proofer.failed_tests).to eq []
	end
	it 'fails on redirects if not following' do
	linkWithRedirectFilepath = "#{FIXTURES_DIR}/links/linkWithRedirect.html"
	proofer = run_proofer(linkWithRedirectFilepath, :typhoeus => { :followlocation => false })
	expect(proofer.failed_tests.first).to match(/failed: 301 No error/)
	end

I also can't seem to reproduce the problem:

require 'html/proofer'

HTML::Proofer.new(["https://html.spec.whatwg.org/"]).run

Is it possibly a temporary connectivity issue on their end?

[doktorbro]

Is it possibly a temporary connectivity issue on their end?

I do not think so. I see the issue since 4 weeks now.

[gjtorikian]

Could something have changed in Ubuntu 14.10? Here seem to me the facts:

• Travis runs Ubuntu 12.04
• I run OS X
• Both myself and Travis yield 200s
• You are on Ubuntu 14.10, and it fails

I am not sure what can be done here. I guess I can check to see if ssl_verifypeer is correctly passed along to Proofer.

[doktorbro]

$ curl -V
curl 7.37.1 (i686-pc-linux-gnu) libcurl/7.37.1 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp 
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

[doktorbro]

I will try to reproduce the issue with Typhoeus only.

[gjtorikian]

I have the exact same version of curl. Odd. Does curl alone work?

[doktorbro]

Curling the specs fails.

curl --url https://html.spec.whatwg.org/multipage/ --verbose
* Hostname was NOT found in DNS cache
*   Trying 208.113.236.128...
* Connected to html.spec.whatwg.org (208.113.236.128) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS alert, Server hello (2):
* error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
* Closing connection 0
curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Curling my user here works.

curl --url https://api.github.com/users/penibelst --verbose
* Hostname was NOT found in DNS cache
*   Trying 192.30.252.136...
* Connected to api.github.com (192.30.252.136) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
*    subject: C=US; ST=California; L=San Francisco; O=GitHub, Inc.; CN=*.github.com
*    start date: 2014-04-08 00:00:00 GMT
*    expire date: 2017-04-12 12:00:00 GMT
*    subjectAltName: api.github.com matched
*    issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
*    SSL certificate verify ok.
# header and content

[doktorbro]

Typhoeus itself returns an error ssl_connect_error

Typhoeus::VERSION
=> "0.6.9"

Typhoeus.get("https://html.spec.whatwg.org/multipage/", { :ssl_verifypeer => false})
=> #<Typhoeus::Response:0x9296cc0 @options={:httpauth_avail=>0, :total_time=>1.230575, :starttransfer_time=>0.0, :appconnect_time=>0.0, :pretransfer_time=>0.0, :connect_time=>0.200696, :namelookup_time=>0.028447, :effective_url=>"https://html.spec.whatwg.org/multipage/", :primary_ip=>"208.113.236.128", :response_code=>0, :request_size=>0, :redirect_count=>0, :return_code=>:ssl_connect_error, :response_headers=>"", :response_body=>"", :debug_info=>#<Ethon::Easy::DebugInfo:0x92b46a8 @messages=[]>}, @request=#<Typhoeus::Request:0x929c10c @base_url="https://html.spec.whatwg.org/multipage/", @original_options={:ssl_verifypeer=>false, :method=>:get}, @options={:ssl_verifypeer=>false, :method=>:get, :headers=>{"User-Agent"=>"Typhoeus - https://github.com/typhoeus/typhoeus"}, :maxredirs=>50}, @response=#<Typhoeus::Response:0x9296cc0 ...>, @on_complete=[], @on_failure=[]>>

@i0rek Any thoughts? Should I open an issue in @typhoeus?

@annevk Any known issues with the certificates at https://html.spec.whatwg.org/ ?

[annevk]

Not with the certificate, but the RC4 cipher DreamHost uses is way old. We are told they will fix it this quarter. See https://www.ssllabs.com/ssltest/analyze.html?d=html.spec.whatwg.org for more information.

[hanshasselberg]

@penibelst Thats not a Typhoeus problem. A workaround would be to set that option: ssl_verifyhost: false, but while playing around with it the problem seems to be solved by whatwg.org.

[doktorbro]

@i0rek ssl_verifyhost: false gives me an error:

NoMethodError: undefined method `to_i' for false:FalseClass

ssl_verifyhost: 0 works.

but while playing around with it the problem seems to be solved

Indeed.

[hanshasselberg]

@penibelst of course you are right regarding the correct value of the ssl_verifyhost option. I am sorry :(.

[gjtorikian]

@penibelst Out of curiosity, how does the https://www.backblaze.com URL work for you? I'm seeing it fail in our company CI, but no one can reproduce it locally. And passing ssl_verifyhost: 0 does not silence the error, even with the recent Typhoeus bump. It's starting to drive me mad!

[doktorbro]

@gjtorikian I can reproduce it locally. Curl only:

curl --url https://www.backblaze.com/ --verbose
* Hostname was NOT found in DNS cache
*   Trying 162.244.56.106...
* Connected to www.backblaze.com (162.244.56.106) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS alert, Server hello (2):
* error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
* Closing connection 0
curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Typhoeus 0.7.1:

Typhoeus.get("https://www.backblaze.com/", { :ssl_verifyhost => 0})
=> #<Typhoeus::Response:0x9229800 @options={:httpauth_avail=>0, :total_time=>0.201073, :starttransfer_time=>0.0, :appconnect_time=>0.0, :pretransfer_time=>0.0, :connect_time=>0.180304, :namelookup_time=>4.3e-05, :effective_url=>"https://www.backblaze.com/", :primary_ip=>"162.244.56.106",:response_code=>0, :request_size=>0, :redirect_count=>0, :return_code=>:ssl_connect_error, :response_headers=>"", :response_body=>"", :debug_info=>#<Ethon::Easy::DebugInfo:0x923a510 @messages=[]>}, @request=#<Typhoeus::Request:0x922a9f8 @base_url="https://www.backblaze.com/", @original_options={:ssl_verifyhost=>0, :method=>:get}, @options={:ssl_verifyhost=>0, :method=>:get, :headers=>{"User-Agent"=>"Typhoeus - https://github.com/typhoeus/typhoeus"}, :maxredirs=>50}, @response=#<Typhoeus::Response:0x9229800 ...>, @on_complete=[], @on_failure=[]>>

[gjtorikian]

@i0rek Sorry to ping you once more! Do you have any insight here as to why ssl_verifyhost: 0 would still cause these misconfigured servers to fail?