Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

misc(logger): 1.3.0, package types, timeDecorate, add tests #6740

Closed
wants to merge 17 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ module.exports = {
collectCoverageFrom: [
'**/lighthouse-core/**/*.js',
'**/lighthouse-cli/**/*.js',
'**/lighthouse-logger/**/*.js',
'!**/test/',
'!**/scripts/',
],
Expand All @@ -19,5 +20,6 @@ module.exports = {
testMatch: [
'**/lighthouse-core/**/*-test.js',
'**/lighthouse-cli/**/*-test.js',
'**/lighthouse-logger/**/*-test.js',
],
};
240 changes: 206 additions & 34 deletions lighthouse-logger/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,42 @@

const debug = require('debug');
const marky = require('marky');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a type file for marky in types/marky? It looks like it should be fairly minimal.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yah?
It looks like it's basically

declare module 'marky' {
  class Marky {
    static mark(name: string): void;
    static stop(name: string): void;
    static getEntries(): PerformanceEntry[];
    static clear(): void;
  }
  export = Marky;
}

or whatever of the many variations that would work in types/marky/index.d.ts


const EventEmitter = require('events').EventEmitter;

/**
* @template T
* @typedef {import('./type-helpers').IsFunction<T>} IsFunction
*/

/**
* @template Class
* @template {*[]} Args
* @typedef {{
* msg: string | ((this: Class, ...args: Args) => string),
* id?: string | ((this: Class, ...args: Args) => string),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if we drop the dynamic messages and only support strings here (at least for now)? It would make this file simpler to read, simpler for type checking, and reduces the non-local reasoning needed where the timeDecorateClass calls are located.

AFAICT, the only places we need dynamically created strings are for

  • running beforePass/pass/afterPass for a gatherer, and
  • running an audit
  • computing a computed artifact.

Due to the structure of gatherRunner there's no good place to decorate the gatherer functions, so #6515 only included dynamic messages for last two. We could do those with manual log.time()/log.timeEnd() calls (only two pairs of them needed) and be able to get rid of a lot of machinery in here.

And then at a later date we can always add support if we find we want it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'll be most immediately needed by #6410, since each call to sendCommand has a timing with an id derived from the input. As soon as this lands I'll be pushing that issue forward again. With that in mind, should this stay?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, yeah, that's useful enough that I guess we should probably keep it

* timeStartLogLevel?: LogAction,
* timeEndLogLevel?: LogAction,
* }} TimeDecorateOpts
*/

/**
* @typedef {'verbose'|'error'|'silent'|'log'|'info'|'warn'} LogLevel
*/
/**
* @typedef {'verbose'|'error'|'log'|'warn'} LogAction
*/

/**
* @typedef {{
* msg: string,
* id: string,
* args?: any[],
* }} Status
*/

const isWindows = process.platform === 'win32';

// process.browser is set when browserify'd via the `process` npm module
// @ts-ignore - process.browser is set when browserify'd via the `process` npm module
const isBrowser = process.browser;

const colors = {
Expand All @@ -23,60 +54,76 @@ const colors = {
magenta: isBrowser ? 'palevioletred' : 5,
};

// whitelist non-red/yellow colors for debug()
// @ts-ignore - whitelist non-red/yellow colors for debug(). not public
debug.colors = [colors.cyan, colors.green, colors.blue, colors.magenta];

class Emitter extends EventEmitter {
/**
* Fires off all status updates. Listen with
* `require('lib/log').events.addListener('status', callback)`
* @param {string} title
* @param {string|Error} message
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

* @param {!Array<*>} argsArray
*/
issueStatus(title, argsArray) {
issueStatus(title, message, argsArray) {
if (title === 'status' || title === 'statusEnd') {
this.emit(title, [title, ...argsArray]);
this.emit(title, [title, message, ...argsArray]);
}
}

/**
* Fires off all warnings. Listen with
* `require('lib/log').events.addListener('warning', callback)`
* @param {string} title
* @param {string|Error} message
* @param {!Array<*>} argsArray
*/
issueWarning(title, argsArray) {
this.emit('warning', [title, ...argsArray]);
issueWarning(title, message, argsArray) {
this.emit('warning', [title, message, ...argsArray]);
}
}

/** @type {Record<string, debug.IDebugger>} */
const loggersByTitle = {};

const loggingBufferColumns = 25;

/** @type {LogLevel} */
let level_;

class Log {
static _logToStdErr(title, argsArray) {
/**
* @param {string} title
* @param {string|Error} message
* @param {Array<any>} argsArray
*/
static _logToStdErr(title, message, argsArray) {
const log = Log.loggerfn(title);
log(...argsArray);
log(message, ...argsArray);
}

/**
* @param {string} title
*/
static loggerfn(title) {
let log = loggersByTitle[title];
if (!log) {
log = debug(title);
loggersByTitle[title] = log;
// errors with red, warnings with yellow.
if (title.endsWith('error')) {
// @ts-ignore
log.color = colors.red;
} else if (title.endsWith('warn')) {
// @ts-ignore
log.color = colors.yellow;
}
}
return log;
}

/**
* @param {string} level
* @param {LogLevel} level
*/
static setLevel(level) {
level_ = level;
Expand All @@ -99,16 +146,16 @@ class Log {
* A simple formatting utility for event logging.
* @param {string} prefix
* @param {!Object} data A JSON-serializable object of event data to log.
* @param {string=} level Optional logging level. Defaults to 'log'.
* @param {LogLevel=} level Optional logging level. Defaults to 'log'.
*/
static formatProtocol(prefix, data, level) {
const columns = (!process || process.browser) ? Infinity : process.stdout.columns;
const columns = isBrowser ? Infinity : (process.stdout.columns || Infinity);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice one

const method = data.method || '?????';
const maxLength = columns - method.length - prefix.length - loggingBufferColumns;
// IO.read blacklisted here to avoid logging megabytes of trace data
const snippet = (data.params && method !== 'IO.read') ?
JSON.stringify(data.params).substr(0, maxLength) : '';
Log._logToStdErr(`${prefix}:${level || ''}`, [method, snippet]);
Log._logToStdErr(`${prefix}:${level || ''}`, method, [snippet]);
}

/**
Expand All @@ -118,33 +165,148 @@ class Log {
return level_ === 'verbose';
}

static time({msg, id, args = []}, level = 'log') {
/**
* @param {Status} status
* @param {LogAction} level
*/
static time({msg, id, args}, level = 'log') {
marky.mark(id);
Log[level]('status', msg, ...args);
Log[level]('status', msg, ...(args || []));
}

static timeEnd({msg, id, args = []}, level = 'verbose') {
Log[level]('statusEnd', msg, ...args);
/**
* @param {Status} status
* @param {LogAction} level
*/
static timeEnd({msg, id, args}, level = 'verbose') {
Log[level]('statusEnd', msg, ...(args || []));
marky.stop(id);
}

static log(title, ...args) {
Log.events.issueStatus(title, args);
return Log._logToStdErr(title, args);
/* eslint-disable no-invalid-this */
/**
* Decorates a function, calling time/timeEnd before/after calling the original function.
* @template {*} R
* @template {*[]} Args
* @param {(...args: Args) => R} originalFn
* @param {TimeDecorateOpts<any, Args>} opts
* @return {(...args: Args) => R}
*/
static timeDecorate(originalFn, opts) {
/**
* @type {(_this: *, args: Args) => string}
*/
const computeMsg = (_this, args) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this fn be as simple as:

  const computeMsg = (_this, args) => {
    if (typeof opts.msg === 'string') return opts.msg;
    if (typeof opts.msg === 'function') return opts.msg.apply(_this, args);
    throw new Error('expected msg');
  };

and same with computeId()

checking that the first two cases don't return something falsy seems.. unnecessary to me, but w/e

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh ya for sure

this only made sense before I figured out how to strongly type all this. had to verify this at runtime. but now TSC does the heavy lifting here :)

if (typeof opts.msg === 'string') return opts.msg;
if (typeof opts.msg === 'function') return opts.msg.apply(_this, args);
throw new Error('expected msg');
};

/**
* @type {(_this: *, args: Args) => string}
*/
const computeId = (_this, args) => {
if (typeof opts.id === 'string') return opts.id;
if (typeof opts.id === 'function') return opts.id.apply(_this, args);
return `lh:${originalFn.name}`;
};

const timeStartLogLevel = opts.timeStartLogLevel || 'log';
const timeEndLogLevel = opts.timeEndLogLevel || 'verbose';

/**
* @type {(this: *, ...args: Args) => R}
*/
const fn = function timeDecoratedFn(...args) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drop the const fn =, or is there a reason for it?

const status = {msg: computeMsg(this, args), id: computeId(this, args)};
Log.time(status, timeStartLogLevel);

let result;
try {
result = originalFn.apply(this, args);
} catch (err) {
Log.timeEnd(status, timeEndLogLevel);
throw err;
}

if (result && typeof result.then === 'function') {
return result.then((/** @type {any} */ value) => {
Log.timeEnd(status, timeEndLogLevel);
return value;
}).catch((/** @type {any} */ err) => {
Log.timeEnd(status, timeEndLogLevel);
throw err;
});
} else {
Log.timeEnd(status, timeEndLogLevel);
return result;
}
};
return fn;
}
/* eslint-enable no-invalid-this */

static warn(title, ...args) {
Log.events.issueWarning(title, args);
return Log._logToStdErr(`${title}:warn`, args);
/**
* Decorates (like timeDecorate) specified methods of a class.
* If decorating instance methods, use the class's prototype.
* If decorating static methods, use the class directly.
* @template {Object|Function} Class
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we be more specific? This is just any

Copy link
Collaborator Author

@connorjclark connorjclark Dec 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to accept anything newable (classes, so basically any function) or objects (prototypes). I suppose objects and functions covers everything :P

What about something silly like this?

* @template {keyof Class} Prop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be limited to properties that are functions? E.g. you can pass a number property in (and it will accept any parameters for msg and id)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this part guards against non-functions:

* @param {{[key in Prop]: TimeDecorateOpts<Class, Parameters<IsFunction<Class[key]>>>}} methods

IsFunction<T> resolves to never if T does not extend Function.

* @param {Class} klass
* @param {{[key in Prop]: TimeDecorateOpts<Class, Parameters<IsFunction<Class[key]>>>}} methods
*/
static timeDecorateClass(klass, methods) {
for (const [method, opts] of Object.entries(methods)) {
if (!opts.id) {
const className = (typeof klass === 'function' ? klass : klass.constructor).name;
opts.id = `lh:${className}:${method}`;
}
/** @type {IsFunction<Class[typeof method]>} */
const original = klass[method];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned that basically everything in this and the above function are typed as any :)

I wonder if there's something simpler (or defining more in a d.ts file if necessary) to get us some constraints enforced. Or we go the opposite, keeping the interface as strict as possible and going explicitly any inside (this is partially where timeDecorate has gone, at least with the return value).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. My main goal was to only allow property names of functions of klass in methods. The current typing achieves that, but the local typing is a bit out of my reach (or TypeScript's?). But the ol' gut type checker seems right to me (AKA it looks correct).

if (!original) {
throw new Error(`Cannot decorate non-existent method ${method}`);
}
klass[method] = Log.timeDecorate(original, opts);
}
}

static error(title, ...args) {
return Log._logToStdErr(`${title}:error`, args);
/**
* @param {string} title
* @param {string|Error} message
* @param {Array<any>} args
*/
static log(title, message, ...args) {
Log.events.issueStatus(title, message, args);
Log._logToStdErr(title, message, args);
}

static verbose(title, ...args) {
Log.events.issueStatus(title, args);
return Log._logToStdErr(`${title}:verbose`, args);
/**
* @param {string} title
* @param {string|Error} message
* @param {Array<any>} args
*/
static warn(title, message, ...args) {
Log.events.issueWarning(title, message, args);
Log._logToStdErr(`${title}:warn`, message, args);
}

/**
* @param {string} title
* @param {string|Error} message
* @param {Array<any>} args
*/
static error(title, message, ...args) {
Log._logToStdErr(`${title}:error`, message, args);
}

/**
* @param {string} title
* @param {string|Error} message
* @param {Array<any>} args
*/
static verbose(title, message, ...args) {
Log.events.issueStatus(title, message, args);
Log._logToStdErr(`${title}:verbose`, message, args);
}

/**
Expand Down Expand Up @@ -228,14 +390,24 @@ class Log {
static get doubleLightHorizontal() {
return '──';
}

/**
* @return {PerformanceEntry[]}
*/
static takeTimeEntries() {
const entries = marky.getEntries();
marky.clear();
return entries;
}

/**
* @return {PerformanceEntry[]}
*/
static getTimeEntries() {
return marky.getEntries();
}
}

Log.events = new Emitter();
Log.takeTimeEntries = () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why were these declared here before? I can't remember. Was it important? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, afaict there was no reason for it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there was a good reason. It only blames to me because I split it off from the mega-PR :)

Seems fine to me!

const entries = marky.getEntries();
marky.clear();
return entries;
};
Log.getTimeEntries = () => marky.getEntries();

module.exports = Log;
2 changes: 1 addition & 1 deletion lighthouse-logger/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "lighthouse-logger",
"version": "1.2.0",
"version": "1.3.0",
"license": "Apache-2.0",
"dependencies": {
"debug": "^2.6.8",
Expand Down
Loading