Mesos: Prevent requests from using chardet

[GregBowyer]

Requests internally (and irritatingly imho) bundles chardet. It uses this library to figure out the encoding for a given response if there are no suitable HTTP headers detailing the character set of the response.

This library is pathologically slow (its loop heavy) on any python implementation not running on top of a VM. There are better implementations of chardet for vanilla pythons but the requests library authors wish to avoid bundling C extensions. https://github.com/kennethreitz/requests/issues/2359

Taking a leaf from opentable, we now force the encoding of the mesos response to always be UTF8 which avoids requests pulling large mesos payloads through chardet and prevents the datadog agent taking large amounts of CPU time to run the check.

[talwai]

Hi @GregBowyer , thanks for contributing! Looks like a worthwhile change, would you in fact be able to provide some benchmarks on performance improvement as a result of your changes to the check? Agent Developer Mode might be helpful here.

[GregBowyer]

Hi @GregBowyer , thanks for contributing! Looks like a worthwhile change, would you in fact be able to provide some benchmarks on performance improvement as a result of your changes to the check? Agent Developer Mode might be helpful here.

I could but I would prefer not to, this gave us serious performance issues in our datacenter. Essentially datadog no-data alarms for all our monitors would start firing with the mesos check enabled, which comes through pagerduty which wakes our support staff up :P.

[talwai]

No worries, and sincere apologies for any rude wake up calls this caused.
We'll profile the improvements on our end, and consider this for an
upcoming release

On Fri, Jan 8, 2016 at 4:22 PM Greg Bowyer [email protected] wrote:

Hi @GregBowyer https://github.com/GregBowyer , thanks for contributing!
Looks like a worthwhile change, would you in fact be able to provide some
benchmarks on performance improvement as a result of your changes to the
check? Agent Developer Mode might be helpful here.

I could but I would prefer not to, this gave us serious performance issues
in our datacenter. Essentially datadog no-data alarms for all our monitors
would start firing with the mesos check enabled, which comes through
pagerduty which wakes our support staff up :P.

—
Reply to this email directly or view it on GitHub
#2192 (comment).