Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner: fix global before not called when no global test exists #48877

Merged
merged 5 commits into from
Jul 24, 2023
Merged

test_runner: fix global before not called when no global test exists #48877

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2969867
test_runner: fix global before not called when no global test exists
rluvaton Jul 21, 2023
99786ae
add test
rluvaton Jul 22, 2023
5b2c5d6
remove ?.
rluvaton Jul 23, 2023
6c5a5af
Update test/fixtures/test-runner/output/hooks-with-no-global-test.js
rluvaton Jul 23, 2023
f2c551c
remove ?.
rluvaton Jul 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions lib/internal/test_runner/test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -821,6 +821,10 @@ class Suite extends Test {
return;
}

if (this.parent?.hooks.before.length > 0) {
rluvaton marked this conversation as resolved.
Show resolved Hide resolved
await this.parent.runHook('before', this.parent?.getRunArgs());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we reach here this.parent will have a value (and we already call runHook without the optional chaining)

Suggested change
await this.parent.runHook('before', this.parent?.getRunArgs());
await this.parent.runHook('before', this.parent.getRunArgs());

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect this to replace the call to this.runHook('before', hookArgs);, and also for the same implementation for beforeEach, afterEach and after

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't for after as if there are multiple describe we only need to run for the last one after all test completes

Copy link
Member Author

@rluvaton rluvaton Jul 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you expect that? Current describe hooks have different hookArgs that the parent (which I think is why some tests are failing when I run with the current tests args)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess It is just surprising that the before and after hooks are not "symmetric"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is, the first describe should run the before and the last describe should run the after...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would simply expect to also see this.parent.runHook('after') - but I guess I am missing something

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's take this test for example:

const {describe, it, before, after} = require("node:test");


describe('top desc', () => {
  before(() => {
    console.log('before top desc');
  });

  after(() => {
    console.log('after top desc');
  });

  describe('inner describe 1', () => {
    before(() => {
      console.log('before inner 1 desc');
    });

    after(() => {
      console.log('after inner 1 desc');
    });

    it('inner it 1', () => {
      console.log('inner it 1');
    });
  });

  describe('inner describe 2', () => {
    before(() => {
      console.log('before inner 2 desc');
    });

    after(() => {
      console.log('after inner 2 desc');
    });

    it('inner it 1', () => {
      console.log('inner it 2');
    });
  });
});

the expected log is:

before top desc
before inner 1 desc
inner it 1
after inner 1 desc
before inner 2 desc
inner it 2
after inner 2 desc
after top desc <-----

if I add this.parent.runHook('after') as well the output will be:

before top desc
before inner 1 desc
inner it 1
after top desc <-----
after inner 1 desc
before inner 2 desc
inner it 2
after inner 2 desc

}

await this.runHook('before', hookArgs);

const stopPromise = stopTest(this.timeout, this.signal);
Expand Down
84 changes: 84 additions & 0 deletions test/fixtures/test-runner/output/hooks-with-no-global-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
'use strict';
const { test, describe, it, before, after, beforeEach, afterEach } = require('node:test');
const assert = require("assert");

// This file should not have any global tests to reproduce bug #48844
const testArr = [];

before(() => testArr.push('global before'));
after(() => {
testArr.push('global after');

try {
assert.deepStrictEqual(testArr, [
'global before',
'describe before',

'describe beforeEach',
'describe it 1',
'describe afterEach',

'describe beforeEach',
'describe test 2',
'describe afterEach',

'describe nested before',

'describe beforeEach',
'describe nested beforeEach',
'describe nested it 1',
'describe afterEach',
'describe nested afterEach',
Comment on lines +30 to +31
Copy link
Member Author

@rluvaton rluvaton Jul 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems really weird that we executing top describe afterEach before the nested one...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see #48736 (comment) also talked about afterEach...


'describe beforeEach',
'describe nested beforeEach',
'describe nested test 2',
'describe afterEach',
'describe nested afterEach',

'describe nested after',
'describe after',
'global after',
]);
} catch (e) {
// TODO(rluvaton): remove the try catch after #48867 is fixed
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because of #48867 we must use process.exit as otherwise, it won't fail the test

console.error(e);
process.exit(1);
}
});

describe('describe hooks with no global tests', () => {
before(() => {
testArr.push('describe before');
});
after(()=> {
testArr.push('describe after');
});
beforeEach(() => {
testArr.push('describe beforeEach');
});
afterEach(() => {
testArr.push('describe afterEach');
});

it('1', () => testArr.push('describe it 1'));
test('2', () => testArr.push('describe test 2'));

describe('nested', () => {
before(() => {
testArr.push('describe nested before')
});
after(() => {
testArr.push('describe nested after')
});
beforeEach(() => {
testArr.push('describe nested beforeEach')
});
afterEach(() => {
testArr.push('describe nested afterEach')
});

it('nested 1', () => testArr.push('describe nested it 1'));
test('nested 2', () => testArr.push('describe nested test 2'));
});
});
rluvaton marked this conversation as resolved.
Show resolved Hide resolved
44 changes: 44 additions & 0 deletions test/fixtures/test-runner/output/hooks-with-no-global-test.snapshot
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
TAP version 13
# Subtest: describe hooks with no global tests
# Subtest: 1
ok 1 - 1
---
duration_ms: *
...
# Subtest: 2
ok 2 - 2
---
duration_ms: *
...
# Subtest: nested
# Subtest: nested 1
ok 1 - nested 1
---
duration_ms: *
...
# Subtest: nested 2
ok 2 - nested 2
---
duration_ms: *
...
1..2
ok 3 - nested
---
duration_ms: *
type: 'suite'
...
1..3
ok 1 - describe hooks with no global tests
---
duration_ms: *
type: 'suite'
...
1..1
# tests 4
# suites 2
# pass 4
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms *
1 change: 1 addition & 0 deletions test/parallel/test-runner-output.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ const tests = [
{ name: 'test-runner/output/describe_it.js' },
{ name: 'test-runner/output/describe_nested.js' },
{ name: 'test-runner/output/hooks.js' },
{ name: 'test-runner/output/hooks-with-no-global-test.js' },
{ name: 'test-runner/output/no_refs.js' },
{ name: 'test-runner/output/no_tests.js' },
{ name: 'test-runner/output/only_tests.js' },
Expand Down