Skip to content

Commit

Permalink
Docs: Testing environment guide corrected. [skip ci]
Browse files Browse the repository at this point in the history
  • Loading branch information
@AnnaTomanek
AnnaTomanek committed Sep 9, 2019
1 parent 98ec4b5 commit a396aeb
Showing 1 changed file with 26 additions and 26 deletions.
52 changes: 26 additions & 26 deletions docs/framework/guides/contributing/testing-environment.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,51 +11,51 @@ Before reading this article we recommend getting familiar with the CKEditor 5 {@

The CKEditor 5 testing environment uses a popular setup with [Karma](https://karma-runner.github.io), [webpack](https://webpack.github.io/), [babel-loader](https://github.com/babel/babel-loader) and [Istanbul](https://github.com/gotwarlost/istanbul). We created some [npm scripts](https://docs.npmjs.com/cli/run-script) which glue all these pieces and special requirements for CKEditor together.

Each CKEditor package has its own tests suite (see for example the [engine's tests](https://github.com/ckeditor/ckeditor5-engine/tree/master/tests)), however, the test runner is available in the [`ckeditor5`](https://github.com/ckeditor/ckeditor5) package which is the central development environment. The actual code of the test runner is implemented in the [`@ckeditor/ckeditor5-dev-tests`](https://www.npmjs.com/package/@ckeditor/ckeditor5-dev-tests) package and can be easily reused outside of `ckeditor5`.
Each CKEditor 5 package has its own tests suite (see for example the [engine's tests](https://github.com/ckeditor/ckeditor5-engine/tree/master/tests)), however, the test runner is available in the [`ckeditor5`](https://github.com/ckeditor/ckeditor5) package which is the central development environment. The actual code of the test runner is implemented in the [`@ckeditor/ckeditor5-dev-tests`](https://www.npmjs.com/package/@ckeditor/ckeditor5-dev-tests) package and can be easily reused outside of `ckeditor5`.

## Running automated tests

In order to run the automated tests use the `yarn run test [<args>...]` command.
In order to run the automated tests, use the `yarn run test [<args>...]` command.

It accepts the following arguments (must be passed after the `--` option):
It accepts the following arguments that must be passed after the `--` option:

* `--watch` (alias `-w`) &mdash; Whether to watch the files and execute tests whenever any file changes.
* `--source-map` (alias `-s`) &mdash; Whether to generate useful source maps for the code.
* `--coverage` (alias `-c`) &mdash; Whether to generate code coverage.
* `--verbose` (alias `-v`) &mdash; Allows switching on webpack logs.
* `--files` &mdash; Specifies tests files to run. Accepts a package name or a glob. Read more about the [rules for converting the `--files` option to a glob pattern](https://github.com/ckeditor/ckeditor5-dev/tree/master/packages/ckeditor5-dev-tests#rules-for-converting---files-option-to-glob-pattern).
* `--browsers` &mdash; Browsers which will be used to run the tests. Defaults to `Chrome`.
* `--debug` (alias `-d`) &mdash; Allows specifying custom debug flags. For example, the `--debug engine` option uncomments the `// @if CK_DEBUG_ENGINE //` lines in the code. Note that by default `--debug` is set to `true` even if you did not specify it. This enables the base set of debug logs (`// @if CK_DEBUG //`) which should always be enabled in the testing environment. You can completely turn off the debug mode by setting `--debug false` option.
* `--watch` (alias `-w`) &ndash; Whether to watch the files and execute tests whenever any file changes.
* `--source-map` (alias `-s`) &ndash; Whether to generate useful source maps for the code.
* `--coverage` (alias `-c`) &ndash; Whether to generate code coverage.
* `--verbose` (alias `-v`) &ndash; Allows switching on webpack logs.
* `--files` &ndash; Specifies test files to run. Accepts a package name or a glob. Read more about the [rules for converting the `--files` option to a glob pattern](https://github.com/ckeditor/ckeditor5-dev/tree/master/packages/ckeditor5-dev-tests#rules-for-converting---files-option-to-glob-pattern).
* `--browsers` &ndash; Browsers that will be used to run the tests. Defaults to `Chrome`.
* `--debug` (alias `-d`) &ndash; Allows specifying custom debug flags. For example, the `--debug engine` option uncomments the `// @if CK_DEBUG_ENGINE //` lines in the code. Note that by default `--debug` is set to `true` even if you did not specify it. This enables the base set of debug logs (`// @if CK_DEBUG //`) which should always be enabled in the testing environment. You can completely turn off the debug mode by setting the `--debug false` option.

### Examples

Run all tests with the code coverage check of the [`ckeditor5-core`](https://github.com/ckeditor/ckeditor5-core) package:

```
```bash
yarn run test -c --files=core
```

Run and watch the [engine's `view` namespace tests](https://github.com/ckeditor/ckeditor5-engine/tree/master/tests/view) and all the tests in [`ckeditor5-typing`](https://github.com/ckeditor/ckeditor5-typing):

```
```bash
yarn run test -cw --files=engine/view,typing
```

Run the `bold*.js` tests in the [`ckeditor5-basic-styles`](https://github.com/ckeditor/ckeditor5-basic-styles) package:

```
```bash
yarn run test -cw --files=basic-styles/bold*.js
```

## Running manual tests

In order to start the manual tests server use the `yarn run manual` task.
In order to start the manual tests server, use the `yarn run manual` task.

The task accepts the following options:

* `--source-map` (alias `-s`) &mdash; Whether to generate useful source maps for the code.
* `--additionalLanguages="ar,pl,..."` &mdash; Specifies extra languages to the [CKEditor 5 webpack plugin](https://www.npmjs.com/package/@ckeditor/ckeditor5-dev-webpack-plugin). Check out the {@link features/ui-language UI language guide} to learn more.
* `--debug` (alias `-d`) &mdash; Allows specifying custom debug flags. For example, the `--debug engine` option uncomments the `// @if CK_DEBUG_ENGINE //` lines in the code. Note that by default `--debug` is set to `true` even if you did not specify it. This enables the base set of debug logs (`// @if CK_DEBUG //`) which should always be enabled in the testing environment. You can completely turn off the debug mode by setting `--debug false` option.
* `--source-map` (alias `-s`) &ndash; Whether to generate useful source maps for the code.
* `--additionalLanguages="ar,pl,..."` &ndash; Specifies extra languages to the [CKEditor 5 webpack plugin](https://www.npmjs.com/package/@ckeditor/ckeditor5-dev-webpack-plugin). Check out the {@link features/ui-language UI language guide} to learn more.
* `--debug` (alias `-d`) &ndash; Allows specifying custom debug flags. For example, the `--debug engine` option uncomments the `// @if CK_DEBUG_ENGINE //` lines in the code. Note that by default `--debug` is set to `true` even if you did not specify it. This enables the base set of debug logs (`// @if CK_DEBUG //`) which should always be enabled in the testing environment. You can completely turn off the debug mode by setting the `--debug false` option.

It starts the server available at http://localhost:8125.

Expand All @@ -65,39 +65,39 @@ A manual test consists of 3 files:

* A `<name>.md` file with the test description.
* A `<name>.js` file with the JavaScript part of the test (e.g. the code initializing an editor).
* A `<name>.html` file with the HTML part of the test. It does not need to be an entire HTML page (with the doctype, etc.), it can be just these HTML elements that you want to define.
* A `<name>.html` file with the HTML part of the test. It does not need to be an entire HTML page (with the doctype, etc.). It can include just the HTML elements that you want to define.

All 3 files are combined together and create a single manual test.

Example Markdown file:
An example Markdown file:

```md
## Create a new link

1. Select a fragment of regular text.
1. Select a fragment of the regular text.
2. Click the toolbar "Link" button.
3. Check if the balloon panel attached to the selection appeared.
4. Fill in the "Link URL" input in the panel.
5. Click the "Save" button.
6. Check if the selected text is converted into a link.
```

Example HTML file:
An example HTML file:

```html
<head>
<style>
/*
Some additional styles which this test needs.
And yes – the test builder will merge this tag with the head defined in a template.
And yes – the test builder will merge this tag with the head defined in the template.
*/
</style>
</head>

<div id="editor">...</div>
```

Example JavaScript file:
An example JavaScript file:

```js
/* globals console, window, document */
Expand Down Expand Up @@ -128,10 +128,10 @@ ClassicEditor

## Test suite and CI

To ensure the highest quality, we maintain a complete test suite with a stable 100% of code coverage for each of the packages. As of May 2018, this means over 8000 tests and the number is growing. Since every package is tested separately, we implement lower-level tests for libraries and higher-level tests for end-user features.
To ensure the highest quality, we maintain a complete test suite with a stable 100% of code coverage for each of the packages. As of September 2019, this means over 11000 tests and the number is growing. Since every package is tested separately, we implement lower-level tests for libraries and higher-level tests for end-user features.

Such an extensive test suite requires a proper continuous integration service. We use [Travis CI](https://travis-ci.com/) as a build platform. This service ensures seamless and fast developer experience and allow us to focus on the job.
Such an extensive test suite requires a proper continuous integration service. We use [Travis CI](https://travis-ci.com/) as a build platform. This service ensures seamless and fast developer experience and allows us to focus on the job.

Besides automated tests, we also maintain a smaller set of manual tests. They help us verify whether something unexpected happens that might have been missed by the automated tests.

When proposing a pull request make sure to add test(s) which verifies it. Every code change should be accompanied by a test which proves that it is needed. Such a strict approach to testing ensures that we have not only 100% of code coverage (which is quite easy to achieve and gives only illusory safety) but also a high level of coverage for cases which we failed to notice initially (and might do that again in the future).
When proposing a pull request make sure to add test(s) that verifies it. Every code change should be accompanied by a test which proves that it is needed. Such a strict approach to testing ensures that we have not only 100% of code coverage (which is quite easy to achieve and gives only illusory safety) but also a high level of coverage for cases that we failed to notice initially (and might do that again in the future).

0 comments on commit a396aeb

Please sign in to comment.