Skip to content
This repository has been archived by the owner on Dec 8, 2024. It is now read-only.

/* istanbul ignore next */ still damages/changes Function.toString() output #856

Open
GerHobbelt opened this issue Dec 25, 2017 · 2 comments

Comments

@GerHobbelt
Copy link

GerHobbelt commented Dec 25, 2017

One can use /* istanbul ignore next */ to prevent istanbul / nyc from instrumenting the next JS statement/expression, which works well, except when used on a function which will be processed by .toString(f) via, for example, String(f) to obtain the function source code in string format.
Then it turns out the not-instrumented function source code was rewritten after all via some whitespace minification process, resulting in, for example, unit tests checking such String(f) output, FAILING when you run them in a istanbul/nyc environment.

Example: https://github.com/GerHobbelt/jison/blob/master/packages/helpers-lib/tests/tests.js

These assertions all PASS when run in node/mocha:

  /* istanbul ignore next: test functions' code is injected and then crashes the test due to extra code coverage statements having been injected */ 
  it("printFunctionSourceCode (direct)", function () {
    function d(i) { /* mock for linters */ }

    assert.strictEqual(helpers.printFunctionSourceCode(function a(x) { return x; }), "function a(x) { return x; }");
    assert.strictEqual(helpers.printFunctionSourceCode(function (x)  { return x; }), "function (x)  { return x; }");
    assert.strictEqual(helpers.printFunctionSourceCode((x) => { return x; }), "(x) => { return x; }");
    assert.strictEqual(helpers.printFunctionSourceCode((x) => x), "(x) => x");
    assert.strictEqual(helpers.printFunctionSourceCode((x) => (d(1), d(2), x)), "(x) => (d(1), d(2), x)");
  });

  /* istanbul ignore next: test functions' code is injected and then crashes the test due to extra code coverage statements having been injected */ 
  it("printFunctionSourceCode (indirect)", function () {
    function d(i) { /* mock for linters */ }

    var f1 = function a(x) { return x; };
    var f2 = function (x)  { return x; };
    var f3 = (x) => { return x; };
    var f4 = (x) => x;
    var f5 = (x) => (d(1), d(2), x);

    assert.strictEqual(helpers.printFunctionSourceCode(f1), "function a(x) { return x; }");
    assert.strictEqual(helpers.printFunctionSourceCode(f2), "function (x)  { return x; }");
    assert.strictEqual(helpers.printFunctionSourceCode(f3), "(x) => { return x; }");
    assert.strictEqual(helpers.printFunctionSourceCode(f4), "(x) => x");
    assert.strictEqual(helpers.printFunctionSourceCode(f5), "(x) => (d(1), d(2), x)");
  });

while the same tests' code FAILS when executed in nyc+mocha with this error report:

  1) helpers API
       printFunctionSourceCode (direct):

      AssertionError: expected 'function a(x){return x;}' to equal 'function a(x) { return x; }'
      + expected - actual

      -function a(x){return x;}
      +function a(x) { return x; }

      at Context.<anonymous> (tests\tests.js:10:498)

  2) helpers API
       printFunctionSourceCode (indirect):

      AssertionError: expected 'function a(x){return x;}' to equal 'function a(x) { return x; }'
      + expected - actual

      -function a(x){return x;}
      +function a(x) { return x; }

      at Context.<anonymous> (tests\tests.js:10:1323)

which shows the whitespace minification of the functions-under-test by istanbul at work.

(Side note: here's the source code for the executed stringification helper printFunctionSourceCode():

function printFunctionSourceCode(f) {
    var src = String(f);             // <-- turn function source code into JS string
    chkBugger(src);
    return src;
}

AFAICT, this issue is closely related to #310, #674 with related discussion in puppeteer/puppeteer#1054

@GerHobbelt
Copy link
Author

The (TEMPORARY) fix in my own codebase uses run-time detection of istanbul, i.e. a function which checks if my code is executing in an istanbul/nyc environment.

Now the tests PASS in both normal mocha and nyc + mocha modes (compare to the tests in the OP):

  /* istanbul ignore next: test functions' code is injected and then crashes the test due to extra code coverage statements having been injected */ 
  it("printFunctionSourceCode (direct)", function () {
    function d(i) { /* mock for linters */ }

    if (!helpers.detectIstanbulGlobal()) {
      assert.strictEqual(helpers.printFunctionSourceCode(function a(x) { return x; }), "function a(x) { return x; }");
      assert.strictEqual(helpers.printFunctionSourceCode(function (x)  { return x; }), "function (x)  { return x; }");
      assert.strictEqual(helpers.printFunctionSourceCode((x) => { return x; }), "(x) => { return x; }");
      assert.strictEqual(helpers.printFunctionSourceCode((x) => x), "(x) => x");
      assert.strictEqual(helpers.printFunctionSourceCode((x) => (d(1), d(2), x)), "(x) => (d(1), d(2), x)");
      assert.strictEqual(helpers.printFunctionSourceCode(x => x + 7), "x => x + 7");
    } else {
      assert.strictEqual(helpers.printFunctionSourceCode(function a(x) { return x; }), "function a(x){return x;}");
      assert.strictEqual(helpers.printFunctionSourceCode(function (x)  { return x; }), "function (x){return x;}");
      assert.strictEqual(helpers.printFunctionSourceCode((x) => { return x; }), "x=>{return x;}");
      assert.strictEqual(helpers.printFunctionSourceCode((x) => x), "x=>x");
      assert.strictEqual(helpers.printFunctionSourceCode((x) => (d(1), d(2), x)), "x=>(d(1),d(2),x)");
      assert.strictEqual(helpers.printFunctionSourceCode(x => x + 7), "x=>x+7");
    }
  });

  /* istanbul ignore next: test functions' code is injected and then crashes the test due to extra code coverage statements having been injected */ 
  it("printFunctionSourceCode (indirect)", function () {
    function d(i) { /* mock for linters */ }

    var f1 = function a(x) { return x; };
    var f2 = function (x)  { return x; };
    var f3 = (x) => { return x; };
    var f4 = (x) => x;
    var f5 = (x) => (d(1), d(2), x);
    var f6 = x => x + 7;

    if (!helpers.detectIstanbulGlobal()) {
      assert.strictEqual(helpers.printFunctionSourceCode(f1), "function a(x) { return x; }");
      assert.strictEqual(helpers.printFunctionSourceCode(f2), "function (x)  { return x; }");
      assert.strictEqual(helpers.printFunctionSourceCode(f3), "(x) => { return x; }");
      assert.strictEqual(helpers.printFunctionSourceCode(f4), "(x) => x");
      assert.strictEqual(helpers.printFunctionSourceCode(f5), "(x) => (d(1), d(2), x)");
      assert.strictEqual(helpers.printFunctionSourceCode(f6), "x => x + 7");
    } else {
      assert.strictEqual(helpers.printFunctionSourceCode(f1), "function a(x){return x;}");
      assert.strictEqual(helpers.printFunctionSourceCode(f2), "function (x){return x;}");
      assert.strictEqual(helpers.printFunctionSourceCode(f3), "x=>{return x;}");
      assert.strictEqual(helpers.printFunctionSourceCode(f4), "x=>x");
      assert.strictEqual(helpers.printFunctionSourceCode(f5), "x=>(d(1),d(2),x)");
      assert.strictEqual(helpers.printFunctionSourceCode(f6), "x=>x+7");
    } 
  });

while this additional unit test for the istanbul detector helper has been added to make sure the detector itself isn't buggy:

  it("detectIstanbulGlobal", function () {
    if (!helpers.detectIstanbulGlobal()) {
      assert.strictEqual(helpers.detectIstanbulGlobal(), false);
    } else {
      var o = helpers.detectIstanbulGlobal();
      assert.ok(o);
      assert.equal(typeof o, 'object');
      var k = Object.keys(o);
      var kp = k.filter(function pass_paths(el) {
        return el.match(/[\\\/][^\\\/]+$/);
      });
      assert.ok(k.length > 0, "expected 1 or more keys in the istanbul global");
      assert.ok(kp.length > 0, "expected 1 or more paths as keys in the istanbul global");
      var kp = k.filter(function pass_istanbul_file_objects(idx) {
        var el = o[idx];
        return el && el.hash && el.statementMap && el.path;
      });
      assert.ok(kp.length > 0, "expected 1 or more istanbul file coverage objects in the istanbul global");
    }
  });

while the istanbul detector code looks like this:

// code to detect whether we're running under istanbul/nyc coverage environment...
function detectIstanbulGlobal() {
    const gcv = "__coverage__";
    const globalvar = new Function('return this')();
    var coverage = globalvar[gcv];
    return coverage || false;
}

export default detectIstanbulGlobal;

which was derived from the nyc instrument output of a test file.

Hope this helps someone out there while this issue is pending/open for istanbul...

GerHobbelt added a commit to GerHobbelt/jison that referenced this issue Dec 25, 2017
… provide unit tests to ensure both the istanbul run-time detector helper (`detectIstanbulGlobal()`) and the function source code stringifier/extractor helpers are on their best behaviour.

Also fix the function code extractor to cope with the arrow function format `x => expression`.
GerHobbelt added a commit to GerHobbelt/jison that referenced this issue Dec 25, 2017
…anbul` environment. This also takes care of gotwarlost/istanbul#856 until it's fixed in the mainline there.

- fix: when both JSON5 and JISON input modes barf a hairball, assume the most important error is the JISON one (show that one first!), while it MAY be a JSON5 format error that triggered it (show that one last!).   Also check for common JISON errors which are obviously never triggered by any odd JSON5 input format error: when we encounter such an error here, we don't confuse matters and forget about the JSON5 fail as it's irrelevant.
@tex0l
Copy link

tex0l commented May 3, 2018

I confirm I have this bug, thanks for the workaround.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants