Web_log doesn't support response times in nanoseconds

[alibo]

Plugin web_log only supports response times in microseconds. In order to convert different types of units to microseconds, it provides an extra option (called time_multiplier) to multiply the given number. However, it only supports integer type. So if the response time is in nanoseconds, I can't set time_multiplier to 0.001.

Could you please support the float type for option time_multiplier as well?

https://github.com/firehol/netdata/blob/master/conf.d/python.d/web_log.conf#L87

#     custom_log_format:                  # define a custom log format
#          time_multiplier: 1000000       # type <int> - convert time to microseconds

https://github.com/firehol/netdata/blob/master/python.d/web_log.chart.py#L632

        resp_time_func = self.configuration.get('custom_log_format', dict()).get('time_multiplier') or 0

        if not isinstance(resp_time_func, int):
            return find_regex_return(msg='Custom log: "time_multiplier" is not an integer')

[ilyam8]

Hi @alibo

Please check if it works correctly after

https://github.com/firehol/netdata/blob/master/python.d/web_log.chart.py#L632

=>

        if not isinstance(resp_time_func, (int, float)):

[ilyam8]

Nice you opened the issue. I completley forgot about time_multiplierand didn't implement it in go verion.

[Wing924]

Btw, I wonder what's the case we need nanoseconds precision for web log? Do you notice the difference between 1ms and 0.1ms?

[alibo]

@l2isbad It works like a charm :D, thanks a lot :)

Nice you opened the issue. I completely forgot about time_multiplierand didn't implement it in go version.

You're welcome 😁
I'm really looking forward to testing the Go version of this plugin. For load balancers with high loads (10k+ RPS), the latency for parsing logs is about 1s (with SSD). Comparing to many monitoring tools it's really fast, but with Go, I think it would be faster!

Btw, I wonder what's the case we need nanoseconds precision for web log? Do you notice the difference between 1ms and 0.1ms?

@Wing924 You're right, but unfortunately many Go libraries use nanoseconds (e.x https://github.com/labstack/echo/blob/master/middleware/logger.go#L182)

[ilyam8]

It works

Nice! PR?

For load balancers with high loads (10k+ RPS)
testing the Go version

This is nice too.I will ask your help for testing/optimization later.

[alibo]

Nice! PR?

I'll send the PR in a few minutes.

This is nice too.I will ask your help for testing/optimization later.

Sure, I'll be glad to help you :)