HTTP::Client may set an output stream for debugging?

[zfjoy520]

Hi,

May HTTP::Client somewhere to set the debug_output stream in verbose mode?

like pure ruby net/http in:
https://github.com/ruby/ruby/blob/51cfea31e559ce281f7252a2dd0117f61a44912e/lib/net/http.rb#L710

tks.

[asterite]

Could you explain what it does? (apart from opening a serious security hole :-P)

[zfjoy520]

e, I'm a rubyist, and a fresher for Crystal.

I wrote a gem, named 'http-proxy', I mirrored the code to github here:
https://github.com/ruby-ext/http-proxy

as write in README.md

when I use RUBYOPT=-w irb in linux/OSX console for debugging,
run the code in irb console:

>> r = TestModule.order_query(params)

due the settings:

http.set_debug_output(STDERR) if $VERBOSE

in:
https://github.com/ruby-ext/http-proxy/blob/master/lib/http-proxy.rb#L47

the STDERR puts all the network details, such as http header strings, http response:

opening connection to 1.2.3.4:8080...
opened
<- "POST /gateway? HTTP/1.1\r\nContent-Type: application/json;charset=UTF-8\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: */*\r\nUser-Agent: Ruby\r\nConnection: close\r\nHost: 1.2.3.4:8080\r\nContent-Length: 223\r\n\r\n"
<- "{\"aa\":\"11\",\"bb\":\"22\",\"cc\":\"33\"}"
-> "HTTP/1.1 200 OK\r\n"
-> "Content-Type: application/json; charset=utf-8\r\n"
-> "Connection: close\r\n"
-> "Content-Length: 31\r\n"
-> "\r\n"
reading 80 bytes...
-> "{\"response\":{\"success\":false,\"err_code\":\"301\",\"err_desc\":\"ERROR_INVALID_ORDER\"}}"
read 80 bytes
Conn close

the set_debug_output method is defined in ruby core net/http:
https://github.com/ruby/ruby/blob/51cfea31e559ce281f7252a2dd0117f61a44912e/lib/net/http.rb#L710

    def set_debug_output(output)
      warn 'Net::HTTP#set_debug_output called after HTTP started', uplevel: 1 if started?
      @debug_output = output
    en

I search the code for crystal all over, but I can't found anything about how to puts the network logs when debugging.

please help me, thank you.

[asterite]

Yeah, it would be nice to have something like that built in in Crystal's http server.

[jhass]

I wonder if it wouldn't even be nicer to just have an IO that forwards it to two other... oh wait https://crystal-lang.org/api/0.25.1/IO/MultiWriter.html

So now we just need #1330

[zfjoy520]

IO::MultiWriter is awesome, I'll try it later, thanks @jhass .

[asterite]

@zfjoy520 He meant to use that in the HTTP::Client implementation in case debug output is enabled. I don't think you'll be able to do it without changing HTTP::Client's code.

[jhass]

Well specifically I meant allowing the user to pass the socket or rather io to the client would allow this and more without as invasive changes.

[icyleaf]

@zfjoy520 You can use Halite with logger to easy debugging request and response raw or only dump response raw string with #to_raw.

[zfjoy520]

@icyleaf thanks.

[RX14]

There's still a feature request here

[straight-shoota]

I added this to #6011

[straight-shoota]

Since #9543 HTTP::Client.new accepts an IO instance. This allows configuring the connection externally, including debug output through IO::MultiWriter.

require "http"

http_connection = TCPSocket.new("crystal-lang.org", 80)

multi_writer = IO::MultiWriter.new(http_connection, STDOUT)
# stapling is necessary because MultiWriter does not support reads
stapled = IO::Stapled.new(http_connection, multi_writer)

client = HTTP::Client.new(stapled, "crystal-lang.org", 80)

puts client.get("/")

Maybe this could be more enhanced, but I'd wait for the HTTP::Client refactoring.
The basic use case should be covered and this issue can be closed.