recording over HTTPS

[indietyp]

I have been using stubr in my tests and wanted to switch to using the recording for more automated mocking.

My current problem is that I am trying to use the reqwest integration in conjunction with a test server with a self-signed certificate. Whenever I try to connect over TLS I get: "Http(reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("...")), port: None, path: "/api/v2/auth/login", query: None, fragment: None }, source: hyper::Error(Connect, "unsuccessful tunnel") })" when using the proxy. (which is to be expected)

How would I be able to fix the problem?

Thanks for the great library, btw!

[beltram]

Hey ! Thanks for the kind words.

Without further investigation I'm clueless. The standalone recorder is using warp-reverse-proxy so I bet there's some warp config to tweak in order to accept self-signed (or even ignore) certificates.

That being said it's not a being deal, there's a better option. This recorder is more aimed to be used standalone in the cli. Since you are using reqwest there's also a direct integration. You can have an example of it here. It has the drawback of requiring a reqwest ClientBuilder (only way to read both request & response) so you'd maybe have to rewrite your tests a bit. It it also only implemented for blocking ; but implementing it for async should be straightforward.

Let me know if that works out for you 😉

[indietyp]

Thanks for the info. I'll try that tomorrow. I just had a look over the source code, and I think I figured out what was happening:

warp-reverse-proxy is initiated here: https://github.com/indietyp/stubr/blob/main/lib/src/record/standalone/proxy.rs#L20

And warp-reverse-proxy initializes a static reqwest client: https://github.com/danielSanchezQ/warp-reverse-proxy/blob/master/src/lib.rs#L53, so my suggestion is a setting where you can set the client because default reqwest (which is the one that gets initialized - https://github.com/danielSanchezQ/warp-reverse-proxy/blob/master/src/lib.rs#L274) does not support self-signed certificates.

I'll also try tomorrow if that would be a potential solution - and if you are up for it - I'd be happy to create a PR with the necessary changes.

[indietyp]

I have been doing some experimentation (HTTPS proxies are kind of a new thing to me^^), and supporting HTTPS is a bit more involved due to the use of CONNECT. This partially stems from the fact that the reqwest client "abuses" a reverse proxy for a proxy, which works fine with HTTP but breaks down in HTTPS. I am unsure if that is the case - I need to investigate further.

[indietyp]

I found kind of a workaround, which forces HTTP: deploy mitmproxy: mitmweb --mode reverse:<domain> and then change the domain I am trying to connect to to the proxy-server URL. This works, but ofc is a bit more involved.

[beltram]

A little too complicated indeed.
Have you tried the solution I proposed ?

[indietyp]

Yes, kind of. I would need to implement Record for the non-blocking variant. The problem here is twofold: the request and response are evaluated eagerly (from my just look over the whole thing - does that mean every request is sent twice using this approach?), and I would need to create a new trait that supports async.

[beltram]

No it's only sent once: record() takes care of sending the request.
You would have to create another AsyncRecord trait and impl it for reqwest::Client. Have a look at async-trait