Skip to content

Commit

Permalink
Update UI Tests documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
jtpio committed Nov 9, 2021
1 parent 6ea51d7 commit 2ca73f0
Showing 1 changed file with 9 additions and 12 deletions.
21 changes: 9 additions & 12 deletions docs/source/dev_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,20 @@ pytest --cov=ipywidgets ./python/ipywidgets
```

To run the Javascript tests in each package directory:

```sh
yarn test
```

This will run the test suite using `karma` with 'debug' level logging.


# Visual Regression Tests

ipywidgets uses [Galata](https://github.com/jupyterlab/galata) framework for visual regression testing. Galata provides a high level API to interact with JupyterLab UI programmatically and tools for capturing, comparison and report generation.
`ipywidgets` uses the [Galata](https://github.com/jupyterlab/jupyterlab/tree/master/galata) framework for visual regression testing. Galata provides a high level API to programmatically interact with the JupyterLab UI, and tools for taking screenshots and generating test reports.

Galata UI tests are written using typescript and [jest](https://github.com/facebook/jest). ipywidgets UI test suites are located in [ui-tests/tests](./ui-tests/tests) directory.
UI tests are written in TypeScript and run with the Playwright test runner. The test suites are located in the [ui-tests/tests](../../ui-tests/tests) directory.

[ui-tests/tests/widgets.test.ts](./ui-tests/tests/widgets.test.ts) test suite uploads a [notebook](./ui-tests/tests/notebooks/widgets.ipynb) into JupyterLab, runs it cell by cell and takes image captures of cell outputs. Cell outputs of this notebook are widgets of different types. Then, cell output captures are compared to [reference images](./ui-tests/reference-output/) to detect any visual regressions. Test output (diffs, result report) generated by Galata are uploaded to GitHub artifact storage and accessible from GitHub Actions page in `Artifacts` section.
The [main test suite](../../ui-tests/tests/widgets.test.ts) uploads a [notebook](../../ui-tests/tests/notebooks/widgets.ipynb) to JupyterLab, runs it cell by cell and captures a screenshot of the cell outputs. The cell outputs correspond to widgets of different types. The cell outputs captures are then compared to the [reference snapshots](../../ui-tests/widgets.test.ts-snapshots/) to detect any visual regression. The test report (diffs, result, videos) is uploaded to GitHub as an artifact and accessible from the GitHub Actions page in `Artifacts` section.

## Running Tests Locally

Expand All @@ -35,22 +35,19 @@ Galata needs to connect to a running instance of JupyterLab 3 to run UI tests. F

```sh
# in ui-tests directory
yarn run start-jlab
```
Alternatively, you can use the command below to launch JupyterLab which is equivalent to `start-jlab` script:

```sh
jupyter lab --expose-app-in-browser --no-browser --ServerApp.token='' --ServerApp.password='' --ServerApp.disable_check_xsrf='True'
yarn run start
```

Then you can run the `test` script which in turn launches Galata to run UI tests:
Then run the `test` script:

```sh
# in the ui-tests directory
yarn run test
```

You can pass additional arguments to Galata by appending parameters to the command as in `yarn run test -- --no-headless`. For a full list of Galata command-line options see [https://github.com/jupyterlab/galata#command-line-options](https://github.com/jupyterlab/galata#command-line-options).
You can pass additional arguments to `playwright` by appending parameters to the command. For example to run the test in headed mode, `yarn run test --headed`.

Checkout the [Playwright Command Line Reference](https://playwright.dev/docs/test-cli/) for more information about the available command line options.

## Adding new UI tests

Expand Down

0 comments on commit 2ca73f0

Please sign in to comment.