From 2ca73f036324b8f63ba1f5df40167887f7267ba0 Mon Sep 17 00:00:00 2001 From: Jeremy Tuloup Date: Tue, 9 Nov 2021 14:15:23 +0100 Subject: [PATCH] Update UI Tests documentation --- docs/source/dev_testing.md | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/docs/source/dev_testing.md b/docs/source/dev_testing.md index e088e1cdf57..938310a53e2 100644 --- a/docs/source/dev_testing.md +++ b/docs/source/dev_testing.md @@ -7,20 +7,20 @@ pytest --cov=ipywidgets ./python/ipywidgets ``` To run the Javascript tests in each package directory: + ```sh yarn test ``` This will run the test suite using `karma` with 'debug' level logging. - # Visual Regression Tests -ipywidgets uses [Galata](https://github.com/jupyterlab/galata) framework for visual regression testing. Galata provides a high level API to interact with JupyterLab UI programmatically and tools for capturing, comparison and report generation. +`ipywidgets` uses the [Galata](https://github.com/jupyterlab/jupyterlab/tree/master/galata) framework for visual regression testing. Galata provides a high level API to programmatically interact with the JupyterLab UI, and tools for taking screenshots and generating test reports. -Galata UI tests are written using typescript and [jest](https://github.com/facebook/jest). ipywidgets UI test suites are located in [ui-tests/tests](./ui-tests/tests) directory. +UI tests are written in TypeScript and run with the Playwright test runner. The test suites are located in the [ui-tests/tests](../../ui-tests/tests) directory. -[ui-tests/tests/widgets.test.ts](./ui-tests/tests/widgets.test.ts) test suite uploads a [notebook](./ui-tests/tests/notebooks/widgets.ipynb) into JupyterLab, runs it cell by cell and takes image captures of cell outputs. Cell outputs of this notebook are widgets of different types. Then, cell output captures are compared to [reference images](./ui-tests/reference-output/) to detect any visual regressions. Test output (diffs, result report) generated by Galata are uploaded to GitHub artifact storage and accessible from GitHub Actions page in `Artifacts` section. +The [main test suite](../../ui-tests/tests/widgets.test.ts) uploads a [notebook](../../ui-tests/tests/notebooks/widgets.ipynb) to JupyterLab, runs it cell by cell and captures a screenshot of the cell outputs. The cell outputs correspond to widgets of different types. The cell outputs captures are then compared to the [reference snapshots](../../ui-tests/widgets.test.ts-snapshots/) to detect any visual regression. The test report (diffs, result, videos) is uploaded to GitHub as an artifact and accessible from the GitHub Actions page in `Artifacts` section. ## Running Tests Locally @@ -35,22 +35,19 @@ Galata needs to connect to a running instance of JupyterLab 3 to run UI tests. F ```sh # in ui-tests directory -yarn run start-jlab -``` -Alternatively, you can use the command below to launch JupyterLab which is equivalent to `start-jlab` script: - -```sh -jupyter lab --expose-app-in-browser --no-browser --ServerApp.token='' --ServerApp.password='' --ServerApp.disable_check_xsrf='True' +yarn run start ``` -Then you can run the `test` script which in turn launches Galata to run UI tests: +Then run the `test` script: ```sh # in the ui-tests directory yarn run test ``` -You can pass additional arguments to Galata by appending parameters to the command as in `yarn run test -- --no-headless`. For a full list of Galata command-line options see [https://github.com/jupyterlab/galata#command-line-options](https://github.com/jupyterlab/galata#command-line-options). +You can pass additional arguments to `playwright` by appending parameters to the command. For example to run the test in headed mode, `yarn run test --headed`. + +Checkout the [Playwright Command Line Reference](https://playwright.dev/docs/test-cli/) for more information about the available command line options. ## Adding new UI tests