Skip to content

Testing

Jump to bottom
Jay Roebuck edited this page Jan 13, 2023 · 14 revisions

As this project is all about doing things 'the right way', we take our testing seriously. So seriously in fact, that the majority of the code base is dedicated to testing. This is because we have 3 different instances of automated tests to help ensure the quality of our work:

  • front-end unit testing
  • back-end API testing
  • end-to-end (e2e) testing

Front-end unit testing

For these, we use React's testing library (from "@testing-library/react"). The purpose of these tests is to ensure that every component being rendered does so as intended. This means that given certain props (or in some cases none), the component renders the correct information. We must also ensure that certain functions are called when certain buttons are pressed etc.

Basic rendering

Testing presence of text on screen:

it('renders some text', async () => {
  const { findByText } = render(
    <MemoryRouter>
      <Temp />
    </MemoryRouter>
  );

  const titleElement = await findByText("Some text", { exact: false });
  expect(titleElement).toBeInTheDocument();
});

Notice that you can choose whether you want the text to be exact or simply contain some substring (exact is optional, so if you want the exact string you can just use findByText with 1 argument.

Button clicks/function calls

Testing running of a function/button click:

it("when button is clicked, call handleClick", async () => {
  const doTheThing = jest.fn()

  const { findByText } = render(
    <ComponentWithButton
      someProp={someProp}
      onChange={doTheThing}
    />
  );

  const button1 = await findByText("My button text");
  fireEvent.click(button1);

  expect(doTheThing).toBeCalledTimes(1);
});

Mocking data

Another important part of these tests is passing dummy data. Sometimes, you need to mock a request (on pages which make direct API calls), and others you simply need to mock objects.

  • Request mocking:
const goodMock = {
  request: {
    query: ALL_THINGS_QUERY,
    variables: {},
  },
  result: {
    data: {
      allThings: [
        { id: 1, title: "Test Thing 1" },
        { id: 2, title: "Test Thing 2" },
      ],
    },
  },
};

const errorRequestMock = {
  request: {
    query: ALL_THINGS_QUERY,
    variables: {},
  },
  error: new Error(),
};

it("renders component page with good data", async () => {
  const { findByText } = render(
    <MockedProvider mocks={[goodMock]} addTypename={false}>
      <Component />
    </MockedProvider>
  );

  const title = await findByText("Somebody");
  expect(title).toBeInTheDocument();
});

it("renders dev error given a bad request", async () => {
  const { findByText } = render(
    <MockedProvider mocks={[errorRequestMock]} addTypename={false}>
      <Component />
    </MockedProvider>
  );

  const devErrorMessage = await findByText("someWord");
  expect(devErrorMessage).toBeInTheDocument();
});

I actually didn't provide all possible examples because the snippet is already getting long, but you get the idea that you can mock the entire request response as seen above.

  • Basic object mocking:
const someList = [
  { id: 1, title: "Test Item 1" },
  { id: 2, title: "Test Item 2" },
];

it("given items, renders a list of buttons (one for each item)", async () => {
  const { findByText } = render(
    <Selector
      items={someList}
      someUnrelatedFunction={jest.fn()}
    />
  );

  const button1 = await findByText("Test Item 1");
  const button2 = await findByText("Test Item 2");

  expect(button1).toBeInTheDocument();
  expect(button2).toBeInTheDocument();
});

As you can see, in the event where the prop is a known type (in this case, a list of 'Item'), we can simply pass them directly as static variables for testing, just to make sure the component renders properly with the passed info!

Back-end unit testing

For these, we use snapshot testing for Django (from snapshottest.django import TestCase) The purpose of these tests is to make sure our graphql endpoints are returning the proper response given any request. These should be located in a tests folder in the app being tested. In Django, the naming convention for tests is that the test suite file MUST start with 'test'. In our case, we have 2 different test suites for the GraphQL schemas (queries and mutations, named test_queries and test_mutations).

Sometimes, you will need to create backend functions that are not directly part of GraphQL (like helpers or other utils). In this case, to create a new test suite, you can do so following Django's standard.

Snapshot testing is simple and powerful, but can also be useless if we are not careful. The idea is to run the given code and create a snapshot of the output. In the future, the test will not pass if the output of the function is not what has been previously stored in the snapshot. Snapshots are automatically managed by the library, and are created for the first time when a test is first run. In the event where you run a test for the first time and the output is wrong (often because the test itself was badly written), you can update the snapshot by running the test command with an extra --snapshot-update flag. Make sure the output of the snapshot is correct before pushing changes, otherwise the test will be useless!

Basic schema testing

To test a schema, you can simply execute a request using the Graphene test client:

from graphene.test import Client

[...]

class YourModelQueryTestCase(TestCase):  
    fixtures = ["path/to/fixture.json"]

    def setUp(self):  
        self.client = Client(schema)  
        self.maxDiff = None  
  
    def test_all_objects(self):  
        response = self.client.execute("""  
            query {
	            allObjects {
		            id
			        title              
			    }           
			}        
		""")  
        self.assertMatchSnapshot(response)

Then, in the snapshots folder, make sure the new snapshot (snap_<test suite name>) matches the appropriate response for the given request.

Mocking data

As with all testing, we are not actually using production data and thus must create mock data for our tests. This is done using fixtures that are loaded into the testing database. It is important to note that these fixtures, although similar to the seeding data, are locating in different places and are only used for testing.

E2E testing

Finally we have end-to-end testing (aka e2e).

If these commands do not work in the container, try running them from your device.

Unlike the unit tests from previous sections, e2e testing is a form of integration testing, which aims to make sure that all of our components (or units) all work together as expected (now that we know each of them function individually). These tests are completely separate from our frontend source code (as you may have noticed, the e2e tests located in jsuite-frontend/cypress/e2e are written in vanilla JS.) The only reason they are located in the same directory as jsuite-frontend is because they share node packages. Your tests belong in the e2e folder, with a sub folder for whichever app you are working on.

Writing tests

Cypress test suites must all start with the 'describe' function. Optionally, you can define functions to be run before every test in the suite (with 'beforeEach'). Anyway, I will stop with these very specific definitions for now, as the Cypress documentation is quite extensive and of high quality. An important thing to note is that for Cypress tests to work, you must make sure to have ids on your html elements, so that Cypress can access them.

/// <reference types="cypress" />

describe('your page', () => {
	  
	it('prompts the user to select an item', () => {	
		cy.visit('/yourapp')		
		cy.get('[id=select-item-prompt]').should('have.text', 'Please select an item (or insert a new one if you have none!)')	
	})
		
	it('renders a selector button for each item', () => {	
		cy.visit('/yourapp')		
		cy.get('[id=selector-button-1]').should('have.text', 'Cypress Mock Item')		
		cy.get('[id=selector-button-2]').should('have.text', 'Cypress Mock Item 2')	
	})
		
	it('renders item information if selector button pressed', () => {	
		cy.visit('/yourapp')	
		cy.get('[id=selector-button-1]').click()	
		cy.get('[id=item-view]')	
		.invoke('text')
		.should('contain', "Cypress UNIVERSAL item")
		.should('contain', "This is for Cypress e2e tests~")	
	})
})

Here, notice that each tests would make use of several components. For example, the selector buttons and the view are two separate components, and we are checking to see that once a selector button is pressed, the view will display the appropariate item. ***Cypress provides a very cool GUI toolkit which helps you debug your e2e tests and show exactly what is going on and what the tests are looking for, based on the code you wrote. I encourage you to use it if tests are not behaving like you expect them to. ***

Mock data

Since we are testing our apps end-to-end, we must make sure that there is data in the database to query and we know what values to validate. To solve this, the Django container will always make sure to seed the data of all applications. It is therefore important not to change this default data, unless you want to change all other tests accordingly. As seen in the file structure section, each application should have a "fixtures" folder, where the desired seeding data is located. When writing your e2e tests, this data is the data you should be validating, and it will always be the same (have the same id, creation date, etc.)

Test coverage

I hope you are comfortable with the topics of this document, because without proper understanding of our testing workflow it will be difficult to contribute adequately to the project! This is due to the fact that we enforce a certain level of code coverage that tests should cover in order to pass the CI (continuous integration) pipeline. Fortunately, you can easily check your tests' code coverage by running the following:

Clone this wiki locally