forked from kiwix/kiwix-js
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
workflows: add CI workflow for unit tests and support running locally
* Load qunit package from npm, this is the start of a larger transition. ref at kiwix#554. * Update QUnit from 2.3 to 2.12. * Use Karma for the running of unit tests (instead of Nightwatch). While it was possible to use a fake "UI" test to open the QUnit web page with Nightwatch, this had numerous limitations: - relies on the fragile and unsupported DOM scraping to collect results, which breaks between versions. ref kiwix#660. - severely limits debugging information for failing tests. - cannot easily be run or debugged locally from the command-line since as the Nightwatch config was pinned to Sauce Labs, and creating a local configuration is not easy because Nightwatch has a hard requirement for installing and running a WebDriver server first. People usually do not have this installed and is hard to set up. - cannot easily be run in a secure container separate from your personal computer, thus putting personal data at risk. - lacks wider integration and plugins to enrich unit testing, such as test coverage reports. Using Karma means: - We can run 'npm test' locally during development and have it automatically run the tests in headless Firefox and Chrome and report back, all from the command-line. - The same exact same stack is also used in CI with SauceLabs for additional browser coverage (same as before). - It has no external dependencies other than the plain web browser itself. This means if you have a development container (e.g. based on Docker) that has Node.js + Firefox + Chromium, you can run the tests there without exposing anything from your personal computer, besides the current directory. <https://timotijhof.net/posts/2019/protect-yourself-from-npm/> - In a future change, we can plug in karma-coverage to generate a test coverage report, to submit to Codecov or Coveralls. ref kiwix#528. * I have pinned the version of 'http-server' and 'nightwatch' in package.json so that these don't silently upgrade in a way that may introduce security issues or drop compatibility for the environment we currently support. Fixes kiwix#653.
- Loading branch information
Showing
14 changed files
with
6,627 additions
and
5,402 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| name: CI | ||
|
|
||
| on: | ||
| - push | ||
| - pull_request | ||
| # Allow running manually | ||
| - workflow_dispatch | ||
|
|
||
| # A workflow run is made up of one or more jobs that can run sequentially or in parallel | ||
| jobs: | ||
|
|
||
| # This job runs always, including for remote pull requests, and | ||
| # has external dependencies or special permission needs, besides a | ||
| # local install of Node.js, Firefox, and Chromium or Chrome. | ||
| # | ||
| # You can run these same tests locally in your own developer | ||
| # environment via `npm ci && npm test`. | ||
| tests-basic: | ||
|
|
||
| # Packages 'firefox' and 'google-chrome' are pre-installed. | ||
| # | ||
| # https://github.com/actions/virtual-environments/blob/ubuntu20/20201210.0/images/linux/Ubuntu2004-README.md | ||
| runs-on: ubuntu-20.04 | ||
|
|
||
| steps: | ||
| # Clone the repo and checkout the commit for which the workflow was triggered | ||
| - uses: actions/checkout@v2 | ||
|
|
||
| # Install Node.js LTS | ||
| - uses: actions/setup-node@v2 | ||
| with: | ||
| node-version: 10.x | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci | ||
|
|
||
| - name: Run tests | ||
| run: npm test | ||
|
|
||
| # This job only runs when SauceLabs credentials are available | ||
| # (e.g. direct pushes and local non-fork PRs) and is used to | ||
| # run tests on additional operating systems and browsers. | ||
| tests-secure: | ||
| runs-on: ubuntu-20.04 | ||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - uses: actions/setup-node@v2 | ||
| with: | ||
| node-version: 10.x | ||
| - name: Install dependencies | ||
| run: npm ci | ||
|
|
||
| - name: Run tests | ||
| env: | ||
| SAUCE_USERNAME: ${{ secrets.SAUCE_USERNAME }} | ||
| SAUCE_ACCESS_KEY: ${{ secrets.SAUCE_ACCESS_KEY }} | ||
| run: npm run test-unit-saucelabs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,19 +1,29 @@ | ||
| sudo : true | ||
| dist : trusty | ||
| os: linux | ||
| language: node_js | ||
| node_js: | ||
| - "node" | ||
| addons: | ||
| sauce_connect: true | ||
| ssh_known_hosts: download.kiwix.org | ||
| before_script: | ||
| - npm install nightwatch http-server | ||
| - "./node_modules/http-server/bin/http-server . &" | ||
| script: | ||
| # The free account on Sauce does not allow more than 5 concurrent sessions (including the main one) | ||
| # So we separate the recent and old browsers in order to respect this limit. | ||
| # NB : latest edge should be added to this list when Microsoft releases the new Edge based on Chromium | ||
| - "./node_modules/nightwatch/bin/nightwatch -c nightwatch.js --env firefox,chrome" | ||
| - "./node_modules/nightwatch/bin/nightwatch -c nightwatch.js --env edge40,edge44" | ||
| - "./node_modules/nightwatch/bin/nightwatch -c nightwatch.js --env firefox45,chrome58,ie11" | ||
| - pkill node || echo "Node process not running (anymore)" | ||
| node_js: "14" | ||
|
|
||
| jobs: | ||
| fast_finish: true | ||
| include: | ||
|
|
||
| - stage: "Cross-browser testing" | ||
| # This stage only runs when saucelabs credentials are available | ||
| # (e.g. direct pushes and local non-fork PRs) | ||
| # https://docs.travis-ci.com/user/environment-variables/ | ||
| # https://docs.travis-ci.com/user/conditions-v1 | ||
| if: NOT fork | ||
| addons: | ||
| sauce_connect: true | ||
| script: | ||
| - npm run test-unit-saucelabs | ||
| - "./node_modules/.bin/http-server . &" | ||
| - sleep 2 | ||
| - curl "http://localhost:8080" | head | ||
| # The free account on Sauce does not allow more than 5 concurrent sessions (including the main one) | ||
| # So we separate the recent and old browsers in order to respect this limit. | ||
| # REMINDER: Keep this list in sync with the Unit tests, in tests/karma.conf.saucelabs.js | ||
| # NB : latest edge should be added to this list when Microsoft releases the new Edge based on Chromium | ||
| - "./node_modules/.bin/nightwatch -c nightwatch.js --env firefox,chrome" | ||
| - "./node_modules/.bin/nightwatch -c nightwatch.js --env edge40,edge44" | ||
| - "./node_modules/.bin/nightwatch -c nightwatch.js --env firefox45,chrome58,ie11" | ||
| - pkill node || echo "Node process not running (anymore)" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.