Instrumenting Async::HTTP

[trevorturk]

Is your feature request related to a problem? Please describe.

I migrated a Rails application from Puma and Typhoeus to Falcon and Async::HTTP.

New Relic seems to be mostly working well after the switch, but I miss the "Web External" details that I had before on the New Relic dashboard. This would show me the average response time for the external API endpoints I was hitting. I found it useful for setting timeout levels per external API.

(For a little more context, in my app, I'm hitting weather data APIs, and some are slower than others. If I can see the average response time depending on the API hostname, it helps me set appropriate timeouts per data source, to limit my app's autoscaling.)

Only time will tell, but I expect more people to consider using Async::HTTP in the future (I've heard about it in multiple talks at RailsConf 2022 so far) so I would love to see New Relic add support here.

Feature Description

"Web External" instrumentation for the Async::HTTP gem, so I can see average response times etc per external API.

Additional context

Perhaps related to #850, /cc @kaylareopelle who reminded me of this issue during the RailsConf 2022 talk from New Relic.

I'd be happy to help with this feature in any way!

Priority

Really Want. (I've been living without it for a while, but it was useful information that I miss.)

Thank you!

[kaylareopelle]

Hi, @trevorturk! 👋

Thanks for submitting this issue! It was great chatting with you at RailsConf. I could see a spike on this issue fitting in well with #850! I don't believe our team will be able to start work on that until Q3 (approx. October 2022). If you're interested in contributing code to instrument this gem in the meantime, we'd love to assist you!

The first step would be to identify the methods that should be instrumented (like the process in the part of the talk about Ruby's Logger class). What methods are you calling when you use Async::HTTP? Do those methods call something else more central that all external requests pass through?

When you have some ideas, please bring your research to this issue and our team can take a look to verify whether they seem like the right ones.

Thanks again for your submission!

[trevorturk]

Thanks for the reply! I don't think I have the time and/or ability to help before Q3 honestly, and I understand if it's not a top priority in any case. I think you'll have more async users over time, but I think most people still reach for typhoeus at this time, so it's not urgent.

FWIW, I think the hook could be in call here: https://github.com/socketry/async-http/blob/main/lib/async/http/internet.rb#L50 but note also the traces gem which could be leveraged as well: https://github.com/socketry/async-http/blob/main/lib/async/http/client.rb#L146 and https://github.com/socketry/traces -- I believe the traces stuff might be worth investigating. There's been some work done around Datadog, but not yet New Relic, mostly for APM stuff, not the (likely simpler) "web external" stuff I'm more interested in.

Thanks again!

[workato-integration]

https://issues.newrelic.com/browse/NR-150264