Unclear errors handling SSL host verification errors and trying to skip host verificaton

[bjm88]

Hi,
We have a situation where a test service does not have valid ssl certificate, either not signed or self signed. The service also uses client auth to authenticate callers. We need to pass ssl cert and key information for SSL client auth, however we want to bypass remote host verification. Typically in SSL libraries these are separate things, however we have having trouble with savon on this front. Below are some example settings and output.

We also became aware of an error/bug in 2.3 that was suggestion to go back to 2.2.0. We have tried 2.2.0, 2.3.0, and 2.11.0 latest with no luck, and varying behavior and output.

2.11.0 latest, seems this issue originally referenced here is still present..
#491

client = Savon.client(log_level: :debug,
log: true,
filters: [:password],
ssl_cert_file: (Rails.root + 'signed.cer').to_s,
ssl_cert_key_file: ('private.key').to_s,
env_namespace: :soap,
namespace_identifier: :urn,
namespace: 'urn:cdc:iisb:2011',
soap_version: 2,
wsdl: "https://example.com/Service?wsdl",
endpoint: "https://example.com/Service",
open_timeout: 30,
read_timeout: 30)

HTTPI GET request to example.com (excon)
Excon::Errors::SocketError: no implicit conversion of OpenSSL::X509::Certificate into String (TypeError)
from /home/ubuntu/.rvm/gems/ruby-2.1.5/gems/excon-0.45.1/lib/excon/ssl_socket.rb:73:in `read'

if we pass
ssl_verify_mode: :none,
we get
HTTPI GET request to wir.dhswir.org (excon)
HTTPI::SSLError: Unable to verify certificate, please set Excon.defaults[:ssl_ca_path] = path_to_certs, ENV['SSL_CERT_DIR'] = path_to_certs, Excon.defaults[:ssl_ca_file] = path_to_file, ENV['SSL_CERT_FILE'] = path_to_file, Excon.defaults[:ssl_verify_callback] = callback (see OpenSSL::SSL::SSLContext#verify_callback), or Excon.defaults[:ssl_verify_peer] = false (less secure).

If we go back to 2.2.0....without passing ssl_verify_mode :none
we get this
HTTPI GET request to wir.dhswir.org (net_http)
HTTPI::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
from /home/ubuntu/.rvm/gems/ruby-2.1.5/gems/httpi-2.0.2/lib/httpi/adapter/net_http.rb:36:in `rescue in request'

This error is kind of expected, so we want to bypass the host verification so we pass
ssl_verify_mode none...then we get this..
HTTPI::SSLError: SSL_read: ssl handshake failure

I brought this up in this ticket also...
#297

I have found if I go on a local host that trusts this unsigned/self signed remote host things work, passing the sll_cert_file and sll_cert_key_file with client auth and not needing to pass ssl_verify_mode none. It seems to be regardless of version of Savon can't get a combination of not verifying host but also working with ssl client auth to work....

Please help, thank you.

[tjarratt]

Hey @bjm88 I'm sorry that you're having this problem. I can't offer you much by way of assistance, other than to suggest that you spend some time debugging what is actually happening when the handshake occurs, and to maybe try switching to a different http client, since net_http, excon and httpclient all have varying support for enabling / disabling SSL validation. Last I checked some of them did not have the best support for signing requests as you described you were doing with your ssl certificate and private key.

For example, you might want to try this :

client = Savon.client(
  adapter: :httpclient,
  #other properties omitted for clarity
)

One thing I did notice in your errors that you posted was that Excon seems to be taking a ruby object that is an X509 cert and trying to coerce it to a string -- you may want to look into that.

The sad reality is that debugging any SSL problem with Savon is always subtly different, and it's literally impossible for me or anyone else to guess what problem you might be having without direct access to the service you're trying to integrate with. I'd love to be able to wave a magic wand and tell you the correct incantation to make this work, but the reality is that you're going to need to find the time to investigate the root cause of the failure, or decide that Savon just isn't worth the time required.

[bjm88]

Thanks, we will try other adapters. I realize pure ssl is hard to debug, done it too many times. I guess my issue as we show....with same files wget and curl work.....we don't even need soap client to test this, it's pure ssl layer...

So wanted confirmation savon supports client Auth while also bypassing host verification.

Also wanted to understand why this old ticket that is closed around x509 to string is still and issue in 2.11.....2.2 works, 2.3-2.11 fail before handshake. This seems like a different issue that prevents us to get to our curret issue on 2.2

Does Savon or dependent adapterrs require some format, like file vs string if file path vs some x509 object as input? DER (binary) vs PEM (based ASCII) format certs? FYI our certs are PEM

[tjarratt]

You should be passing along a string which is the file path to a PEM encoded certificate, last I checked the docs specify this as do the tests for various classes in Savon.

Unfortunately, no one has shared a reproducible test case that makes this easy to investigate and fix. If you're interested, and can isolate what's wrong, or can write a failing test, I'd love to investigate more since this seems to be harming a lot of people.

[bjm88]

tjarratt - thanks again for the follow ups. I have done a considerable amount of more digging, I think I've isolated the issue and got past it, I would love to hear your thoughts.

Quick Answer for anyone - Yes replace the default Excon adapter with the :httpclient one as suggested, that resolved my issues.

I think other issues exist with version of Excon Savon wants, should be updated. Also I think with Excon the ssl_verify_mode :none doesn't get passed, need to also set
Excon.defaults[:ssl_verify_peer] = false

Should fix both of these in my opinion.

Details...

1. I suspected the Excon lib and perhaps the version of it after your adapter comment. I think Savon should ask for Excon 0.45.1 rather than 0.44.1 given this ticket
   Excon::Errors::SocketError with Ruby 2.2.1 excon/excon#482
   However this didn't immediatley help my situation even when I forced bundle update excon

Reproducing test case
Gemfile
gem 'excon', '0.45.1'
gem 'savon', '~> 2.11.0'

bundle exec rails c

Savon::VERSION returns 2.11.0
client = Savon.client(log_level: :debug,
log: true,
ssl_verify_mode: :none,
ssl_cert_file: (Rails.root + '../security/integrations/wi_iis/wi-signed.cer').to_s,
ssl_cert_key_file: (Rails.root + '../security/integrations/wi_iis/int.key').to_s,
wsdl: "https://wir.dhswir.org/CDC/VaccinationService?wsdl",
endpoint: "https://wir.dhswir.org/CDC/VaccinationService",
open_timeout: 30,
read_timeout: 30)

3 errors I have seen, step by step..
First error is if the host cannot be verified (regardless if you pass ssl_verify_mode :none)
HTTPI::SSLError: Unable to verify certificate, please set Excon.defaults[:ssl_ca_path] = path_to_certs, ENV['SSL_CERT_DIR'] = path_to_certs, Excon.defaults[:ssl_ca_file] = path_to_file, ENV['SSL_CERT_FILE'] = path_to_file, Excon.defaults[:ssl_verify_callback] = callback (see OpenSSL::SSL::SSLContext#verify_callback), or Excon.defaults[:ssl_verify_peer] = false (less secure).

I have found Savon doesn't seem to pass to underlying Excon lib a necessary parameter...so you can set it globally for Excon
Excon.defaults[:ssl_verify_peer] = false

That will get you past first error...then second will come..
Excon::Errors::SocketError: no implicit conversion of OpenSSL::X509::Certificate into String (TypeError)
from /home/ubuntu/.rvm/gems/ruby-2.1.5@caredox/gems/excon-0.44.1/lib/excon/ssl_socket.rb:73:in `read'

I found this may depend on version of Savon 2.1 vs 2.2. vs 2.3 vs 2.11 and also version of Excon 0.44.1 or 0.45.1 (suppose to be fixed in 0.45.1, not sure if Savon not actually using that)

3rd error
Excon::Errors::SocketError: undefined local variable or method `error' for #Excon::SSLSocket:0x000000056c49b8 (NameError)
from /home/ubuntu/.rvm/gems/ruby-2.1.5@caredox/gems/excon-0.45.1/lib/excon/socket.rb:61:in
(also suppose to be fixed in Excon 0.45.1, not sure)

To Verify I was using same Keys in PEM format with WGET utility
wget 'https://wir.dhswir.org/CDC/VaccinationService?wsdl' --certificate=wi-signed.cer --private-key=int.key -O- --no-check-certificate

and all working fine.

The final solution for Savon access was to use the :httpclient and not excon.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue is now closed due to inactivity. If you believe this needs further action, please re-open to discuss.