Alternative for cURL --resolve flag

[JoyceBabu]

Is there an alternative for the --resolve flag of cURL? Setting the Host header alone does not work when the target server is using SNI. In httpie there is a feature request for this, but is pending on an upstream PR.

httpie/cli#99

[ducaale]

This is something we can easily support by making use of reqwest's ClientBuilder::resolve() method.

Would you be interested in opening a PR for adding this feature?

[JoyceBabu]

Sorry, I do not have the enough experience with Rust to undertake something like this.

[iammatthi]

ClientBuilder::resolve() seems to not consider ports.

From reqwest docs:

Since the DNS protocol has no notion of ports, if you wish to send traffic to a particular port you must include this port in the URL itself, any port in the overridden addr will be ignored and traffic sent to the conventional port for the given scheme (e.g. 80 for http).

So would it be okay to use this flag with the --resolve <HOST:ADDRESS> syntax (e.g.: xh example.com --resolve example.com:127.0.0.1)?

[ducaale]

Hmm, that's unfortunate.

I prefer us to stay compatible with cURL's HOST:PORT:ADDRESS format if possible. So if a user passes --resolve example.com:443:127.0.0.1, we rewrite it to resolve("example.com", "127.0.0.1:443".parse()?).

[JoyceBabu]

In the --resolve option, the ip address given in the substitution (resolved) ip address, not the DNS resolver ip address. So, --resolve example.com:443:127.0.0.1 requests curl to use the ip address 127.0.0.1 for any connection to https://example.com. Since the IP address is already mentioned, there is no need to do any further DNS resolution.

For example, if we have a web page at http://example.org that redirects to https://example.com/?sid=12345, and you want to test it locally you can use

curl -L --resolve example.org:80:127.0.0.1  --resolve example.com:443:127.0.0.1 http://example.org

[iammatthi]

I've tried to use the resolve() method, to resolve http://example.com:3000 to 127.0.0.1. By harcoding it I've found that it resolves it correctly with:
client.resolve("example.com", "127.0.0.1:80".parse().unwrap())
but not with:
client.resolve("example.com:3000", "127.0.0.1:80".parse().unwrap()).
In fact I think that the default resolver from reqwest works only with a simple domain name (without port).

An option seems to be to develop a custom resolver by using the dns_resolver method.

[iammatthi]

I've tried to use the resolve() method, to resolve http://example.com:3000 to 127.0.0.1. By harcoding it I've found that it resolves it correctly with:
client.resolve("example.com", "127.0.0.1:80".parse().unwrap())
but not with:
client.resolve("example.com:3000", "127.0.0.1:80".parse().unwrap()).
In fact I think that the default resolver from reqwest works only with a simple domain name (without port).

An option seems to be to develop a custom resolver by using the dns_resolver method.

I meant that I wanted to resolve http://example.com:3000 as 127.0.0.1:3000.

[ducaale]

@iammatthi After rereading your message and the docs, I'm realising that the port number in SocketAddr is always ignored by reqwest.

An option seems to be to develop a custom resolver by using the dns_resolver method.

The resolver passed to ClientBuilder::dns_resolver() doesn't receive a port number, so want can't make it reliably work for redirected requests. Although interestingly enough, changing the port would lead to resolve being called again.

@JoyceBabu How useful do you find a --resolve flag that doesn't take a port number?

xh --resolve=example.org:127.0.0.1 --resolve=example.com:127.0.0.1 http://example.org

[JoyceBabu]

I have never had a situation where I had to resolve the hostname to different ip address based on the port number. But I think it would be better to follow the syntax of curl, even if the port number is silently ignored. This allows compatibility with curl, and at the same time makes it possible to implement this properly, if there ever is a need for it.

[ducaale]

Sounds good.

In that case, let's support both HOST:ADDRESS and HOST:PORT:ADDRESS. And to avoid any potential confusion, we can warn users that we're ignoring the port number in the latter syntax.

[blyxxyz]

You can't usually plug curl flags into xh/HTTPie anyway, so I don't think that kind of compatibility has a lot of upside.

Maybe if we detect the curl syntax we can suggest the port-less syntax in the error message?

If we support it now with broken behavior then implementing it properly later would technically be a compatibility break.

[lapo-luchini]

Please implement in any form, so that feature can be used. 😇

[rivasdiaz]

Can I humbly request a release now that this issue is merged?
I need to use this functionality very often, so I'm eager to try it with xh.

Thanks!

[ducaale]

@rivasdiaz the plan is to cut a new release this weekend. In the meantime, you can build xh from source and test the feature.