Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need a way to get test status in afterEach() method #5292

Open
RudyLu opened this issue Jan 12, 2018 · 28 comments
Open

Need a way to get test status in afterEach() method #5292

RudyLu opened this issue Jan 12, 2018 · 28 comments

Comments

@RudyLu
Copy link

RudyLu commented Jan 12, 2018

Do you want to request a feature or report a bug?
feature request.
What is the current behavior?
Cannot find a way to get test status in afterEach()

What is the expected behavior?
Need a way to get test status, like mocha can do this with this.currentTest.state.

Please provide your exact Jest configuration and mention your Jest, node,
yarn/npm version and operating system.

Jest 22.0.6

@RudyLu RudyLu changed the title Need a way to get test status on afterEach() method Need a way to get test status in afterEach() method Jan 12, 2018
@Peta5000
Copy link

Peta5000 commented Mar 6, 2018

+1
In combination with Puppeteer it would be perfect to get the test status in afterEach().
e.g. we could create a screenshot in afterEach() if status is failed

@michaeljones
Copy link

As a data point, my company's test suite uses the mocha currentTest.state to rollback the current database transaction in a particular way if there was a failure. Seems to be a blocker for switching to Jest though I confess I'm not very experienced with the ins and outs of either our test suite or Jest :)

@jeromemacias
Copy link

I use Browserstack to run my integration test, and I need to flag the browserstack build as complete or fail.
I must be in afterEach and / or afterAll because I need the driver instance currently used (see https://github.com/jeromemacias/nodium/blob/master/src/hook/browserstack.js)

@lefoy
Copy link

lefoy commented Dec 5, 2018

Same as the others, we are using Jest with the TestObject/SauceLabs device farm and we need a way to get the current test status to update the test status on the TestObject website. It would be very useful to have something like this.currentTest in Mocha!

@lefoy
Copy link

lefoy commented Dec 5, 2018

I was able to find a workaround to get the current test status using jasmine custom reports.

const reporter = {
  specDone: async result => {
    console.log(result.status);
    await driver.quit();
  }
};

jasmine.getEnv().addReporter(reporter);

@sergey-siuchanka
Copy link

Is there any progress on that?

@kevin-monteiro
Copy link

Will this feature be implemented in the near future
Need to decide whether to use Jest or not for testing based on this feature availability

@jeysal
Copy link
Contributor

jeysal commented Apr 10, 2019

I believe the category of "passing state into tests/hooks" suggestions is currently not a top priority unfortunately.

Note that we recommend not using any jasmine-specific APIs like one of the comments suggests because jest-jasmine2 will be replaced as a default by jest-circus (likely in the next major).

This seems like something that could be implemented in user land? And made into reusable helpers with something like

test('a', saveTestState(() => {/* ... */}));
afterEach(ifTestFailed(() => {/* ... */}));

@mz026
Copy link

mz026 commented Apr 12, 2019

Hi @jeysal , thanks for the suggestion!
Can you please elaborate more about how to implement the saveTestState and ifTestFailed ?

@jeysal
Copy link
Contributor

jeysal commented Apr 13, 2019

I was thinking about something like:

let lastTestFailed;
export const saveTestState = async (testFn) => {
	try {
		await testFn()
		lastTestFailed = false;
    } catch (err) {
		lastTestFailed = true;
		throw err
	}
}
export const ifTestFailed = async (hookFn) => {
	if(lastTestFailed) await hookFn()
}

Not tested and not complete e.g. because it doesn't take the possible done callback into account, but that's the general idea.

@pplante
Copy link

pplante commented Jan 30, 2020

I found a solution using jest-circus to implement a new testEnvironment, here is my code:

Create a file in the same directory as jest.config.js, named seleniumEnvironment.js

const JSDOMEnvironment = require('jest-environment-jsdom-fifteen')

class SeleniumEnvironment extends JSDOMEnvironment {
  constructor(config, context) {
    super(config, context)

    this.global.hasTestFailures = false
  }

  handleTestEvent(event, state) {
    if (event.name === 'test_fn_failure') {
      this.global.hasTestFailures = true
    }
  }
}

module.exports = SeleniumEnvironment

In jest.config.js I added: testEnvironment: './seleniumEnvironment',

Now in your afterEach/afterAll/ you can detect test failures via this.global.hasTestFailures.

@rgomezp
Copy link

rgomezp commented Feb 1, 2020

I am also looking for a way to do this.

@pplante this didn't work for me. I have also never used jest-circus though. Are there any additional steps needed?

@baruchiro
Copy link

baruchiro commented Feb 14, 2020

@rgomezp You need to define the jest-circus as test runner:
https://github.com/facebook/jest/tree/master/packages/jest-circus

@TimCN
Copy link

TimCN commented Mar 20, 2020

Hack the global.jasmine.currentEnv_.fail works for me.

      describe('Name of the group', () => {

        beforeAll(() => {

          global.__CASE_FAILED__= false

          global.jasmine.currentEnv_.fail = new Proxy(global.jasmine.currentEnv_.fail,{
            apply(target, that, args) {
              global.__CASE__FAILED__ = true
              // you also can record the failed info...
              target.apply(that, args)
              }
            }
          )

        })

        afterAll(async () => {
          if(global.__CASE_FAILED__) {
            console.log("there are some case failed");
            // TODO ...
          }
        })

        it("should xxxx", async () => {
          // TODO ...
          expect(false).toBe(true)
        })
      });

@aesyondu
Copy link

aesyondu commented Apr 11, 2020

I recently had to get access to the test name during the test, so I followed this comment #7774 (comment).

My plan is to use jasmine.currentTest.failedExpectations.length > 0 to identify if there are errors in afterEach()

EDIT: FWIW, it works with playwright, failedExpectations includes TimeoutError from playwright.

  afterEach(() => {
    console.log(jasmine.currentTest)
    console.table(jasmine.currentTest)
  })

    it("asdf", async () => {
      await page.waitForSelector("does-not-exist", {
        timeout: 1000
      })
      // expect(true).toBe(false)
    })

@joshuapinter
Copy link

Leaving our solution here in case it's useful to anybody else.

You can store current spec results in Jasmine and access it in afterEach.

  1. Add a custom Jasmine reporter for specStarted and store the spec results to jasmine.currentTest.

    jasmine.getEnv().addReporter( {
      specStarted: result => jasmine.currentTest = result
    } );

    The unintuitive thing about this is that even though we're storing this in specStarted before the results are in, jasmine.currentTest stores a reference to the result object which will get updated dynamically as the spec runs so when we access it in our afterEach, it will be holding the results of the spec properly.

  2. Check for failedExpectations in your afterEach and take a screenshot if there's been any failures.

    afterEach( async () => {
      if ( jasmine.currentTest.failedExpectations.length > 0 ) { // There has been a failure.
        await driver.takeScreenshot();
      }
    } );

Here's a link to my StackOverflow answer as well: https://stackoverflow.com/a/62557472/293280

@wunnle
Copy link

wunnle commented Jul 17, 2020

How can I access jasmine in the .test.js files? I'm getting 'jasmine' is not defined. when I try to add a reporter.

@joshuapinter
Copy link

How can I access jasmine in the .test.js files? I'm getting 'jasmine' is not defined. when I try to add a reporter.

I added the reporter in our Appium's driver.js file. There, jasmine is automatically setup (by Jest?) as a global. ESLint even complains about it not being defined but it's available at runtime.

@e-hein
Copy link

e-hein commented Aug 21, 2020

as @joshuapinter explained here:

https://stackoverflow.com/a/62557472/293280
you can add an custom reporter to get access.

I created a little helper function:

get-current-spec-result.ts

let currentSpecResult: jasmine.CustomReporterResult;

jasmine.getEnv().addReporter({
  specStarted: (result) => currentSpecResult = result,
});

export function getCurrentSpecResult() {
  return currentSpecResult;
}

so I can import it in any spec and as a side effect, the reporter gets registered (but only if I use it and only once):

get-current-spec-result.spec.ts

import { getCurrentSpecResult } from './get-current-spec-result.ts';

describe('get spec result', () => {
  it('should fail', () => {
    fail('!');
  });

  afterEach(() => {
    const result = getCurrentSpecResult();
    const failed = result.failedExpectations?.pop();
    expect(failed).toBeTruthy();
  });
});

image

wicked? ;-D

@github-actions
Copy link

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the Stale label Feb 25, 2022
@HaveSpacesuit
Copy link

Commenting to prevent the bot from closing this as stale. Would really be good to access test status in after each without having to hack Jasmine.

@github-actions github-actions bot removed the Stale label Feb 28, 2022
@raquelhortab
Copy link

Commenting to prevent the bot from closing this as stale. Would really be good to access test status in after each without having to hack Jasmine.

or without needing jasmine at all

@claytonneal
Copy link

Stunned this feature doesnt exist! Please provide an ETA

@ewangreen
Copy link

ewangreen commented Nov 28, 2022

Also surprised this feature doesn't exist when converting from mocha/chai to jest to make use of its snapshots.

In case it helps anyone, it feels dirty but at least it works post jasmine deprecation.
I've managed to implement it in typescript using jest-circus's test events, storing test names / statuses in NodeEnvironment.global for use in after hooks.
Stripped down example below:

Override handleTestEvent to add test status to global in a custom environment

class CustomEnvironment extends NodeEnvironment {
  handleTestEvent(event: Event) {
    if (!this.global.testStatuses) this.global.testStatuses = {};
    
    switch (event.name) {
      case 'test_fn_failure':
        // event contains info about the test and its parent describe blocks if you need more specific naming.
        // I got a full path by looping through parents until I got to 'ROOT_DESCRIBE_BLOCK'
        const testName = event.test.name;
        this.global.testStatuses[testName] = 'failed'
        break;
        ...
        // I also used test_fn_start and test_fn_success
    }
  }
}

Use event test name to match up in after each hook (in setupFilesAfterEnv)

global.afterEach(() => {
  const testName = expect.getState().currentTestName;
  const match = Object.keys(global.testStatuses).find(
    (item: string) => item === testName
  );
  
  if (match && global.testStatuses[match] === 'failed') { ... }
});

Adding testStatuses to global in global.d.ts

declare global {
  var testStatuses: { [testPath: string]: 'failed' | 'passed' | 'running' };
}

Just note this'll break if test names are not unique in this specific implementation, but could use other identifiers.

@mirobo
Copy link

mirobo commented Oct 19, 2023

Any update on this one? 😥

@mieszko4
Copy link

Just note this'll break if test names are not unique in this specific implementation, but could use other identifiers.

For me the problem is that expect.getState().currentTestName returns full name including all parents (e.g. name of describe), whereas event.test.name is child test name. To fix I had to go through the parents and concatenate strings..

@rgomezp
Copy link

rgomezp commented Nov 16, 2023

This would be great functionality

@stephengaudet
Copy link

+1 Need a way to get test status in afterEach() method

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests