update QUnit, use Karma, make some of CI work locally as well

* Load qunit package from npm. Start of a larger transition,
  ref #554.

* Update QUnit from 2.3 to 2.12.

* Use Karma for the running of unit tests (instead of Nightwatch).

  While it was possible to use a fake "UI" test to run a QUnit
  web page with Nightwatch, this was not ideal. This had numerous
  limitations, such as:

  - relies on fragile an unsupported web scraping to collect results,
    which can easily break between versions.
    ref #660.

  - results provide limited debugging information when a test fails.

  - cannot easily be run locally from the command-line since the
    Nightwatch config is currently written for SauceLabs, and creating
    a local configuration is not easy because Nightwatch has a hard
    requirement on webdriver (which makes sense for UI tests), which
    people usually do not have installed or may be incompatible with the
    current version of their browser. These then also have to be configured
    and started etc.

  - cannot easily be run in a secure container separate from your
    personal computer, thus putting personal data at risk.

  - lacks wider integration and plugins to enrich unit testing,
    such as test coverage reports.

* Using Karma means:
  - You can run 'npm test' locally during development and have it
    automatically run the tests in headless Firefox and Chrome
    and report back, all from the command-line.
  - The same exact runner is also used in CI with SauceLabs
    for additional browser coverage (same as before).
  - It has no external dependencies other than the plain web
    browser itself. This means if you have a development container
    (e.g. based on Docker) that has Node.js + Firefox + Chromium,
    you can run the tests from within there without exposing
    anything from your personal computer, except the current
    working directory.
    <https://timotijhof.net/posts/2019/protect-yourself-from-npm/>
  - In a future change, we can plug in karma-coverage to generate
    a test coverage report, and then submit this to Codecov or
    Coveralls, ref #528.

* I have restructured the Travis CI file so that the cross-browser
  test in SauceLabs are skipped for PRs from forks. Instead, those
  PRs will run the unit tests in local Fireox and Chrome within
  Travis CI. This avoids the confusing failures and still gives
  (some) useful results.

  The tests that Travis CI runs are the same as what users can
  run locally if they invoke `npm test` (after a one-time run
  of `npm install` to fetch dependencies into the local node_modules
  directory).

  This restructuring uses the 'stages' feature which is not
  compatible with the 'deploy' feature. I had converted the
  two deploy stages to use the 'stages' feature as well so that
  these continue to work.

* I have pinned the version of 'http-server' and 'nightwatch'
  in package.json so that these don't silently switch major
  versions that may contain breaking changes.

Fixes #653.

.gitignore

@@ -50,6 +50,7 @@ www-ghdeploy
 /build/
 /scripts/travisci_builder_id_key
 /scripts/secret_files.tar.gz
+/reports/
 
 #Visual Studio
 /.vs/

.travis.yml

@@ -1,30 +1,56 @@
-sudo : true
-dist : trusty
+os: linux
 language: node_js
-node_js:
-  - "node"
+node_js: "14"
+
 addons:
-  sauce_connect: true
   ssh_known_hosts: download.kiwix.org
-before_script:
-    - npm install nightwatch http-server
-    - "./node_modules/http-server/bin/http-server . &"
-script:
-    # The free account on Sauce does not allow more than 5 concurrent sessions (including the main one)
-    # So we separate the recent and old browsers in order to respect this limit.
-    # NB : latest edge should be added to this list when Microsoft releases the new Edge based on Chromium
-    - "./node_modules/nightwatch/bin/nightwatch -c nightwatch.js --env firefox,chrome"
-    - "./node_modules/nightwatch/bin/nightwatch -c nightwatch.js --env edge40,edge44"
-    - "./node_modules/nightwatch/bin/nightwatch -c nightwatch.js --env firefox45,chrome58,ie11"
-    - pkill node || echo "Node process not running (anymore)"
-deploy:
-    - provider: script
-      # Nightly builds (launched by cron)
-      script: sudo apt-get update -q && sudo apt-get --yes install click click-reviewers-tools && ./scripts/setup_travis_env.sh && DISPLAY=:99.0 ./scripts/create_all_packages.sh
-      on:
-          condition: ( "$TRAVIS_EVENT_TYPE" = "cron" )
-    - provider: script
-      # Generation of packages for public releases (launched by a tag)
-      script: sudo apt-get update -q && sudo apt-get --yes install click click-reviewers-tools && ./scripts/setup_travis_env.sh && DISPLAY=:99.0 ./scripts/create_all_packages.sh -t -v ${TRAVIS_TAG}
-      on:
-          tags: true
+
+jobs:
+  fast_finish: true
+  include:
+
+    - stage: "Local testing"
+      # This stage always runs, even for PRs from forks.
+      # It requires no credentials, and uses latest browsers on Linux within CI.
+      # This uses the standard `npm test` command that people can also easily
+      # run locally. That is why we don't skip it, even if credentials are
+      # available, to verify these configurations also.
+      node_js: "10"
+      addons:
+        firefox: latest-esr
+        chrome: stable
+      script: npm test
+
+    - stage: "Cross-browser testing"
+      # This stage only runs when saucelabs credentials are available
+      # (e.g. direct pushes and local non-fork PRs)
+      # https://docs.travis-ci.com/user/environment-variables/
+      # https://docs.travis-ci.com/user/conditions-v1
+      if: NOT fork
+      addons:
+        sauce_connect: true
+      script:
+        - npm run test-unit-saucelabs
+        - "./node_modules/.bin/http-server . &"
+        - sleep 2
+        - curl "http://localhost:8080" | head
+        # The free account on Sauce does not allow more than 5 concurrent sessions (including the main one)
+        # So we separate the recent and old browsers in order to respect this limit.
+        # REMINDER: Keep this list in sync with the Unit tests, in tests/karma.conf.saucelabs.js
+        # NB : latest edge should be added to this list when Microsoft releases the new Edge based on Chromium
+        - "./node_modules/.bin/nightwatch -c nightwatch.js --env firefox,chrome"
+        - "./node_modules/.bin/nightwatch -c nightwatch.js --env edge40,edge44"
+        - "./node_modules/.bin/nightwatch -c nightwatch.js --env firefox45,chrome58,ie11"
+        - pkill node || echo "Node process not running (anymore)"
+
+    # Nightly builds (launched by cron)
+    - stage: "Deploy: Create nightly packages"
+      if: (type IN (cron)) AND NOT fork
+      install: sudo apt-get update -q && sudo apt-get --yes install click click-reviewers-tools
+      script: ./scripts/setup_travis_env.sh && DISPLAY=:99.0 ./scripts/create_all_packages.sh
+
+    # Generation of packages for public releases (launched by a tag)
+    - stage: "Deploy: Create release packages"
+      if: (type = push) AND (tag IS present) AND NOT fork
+      install: sudo apt-get update -q && sudo apt-get --yes install click click-reviewers-tools
+      script: ./scripts/setup_travis_env.sh && DISPLAY=:99.0 ./scripts/create_all_packages.sh -t -v ${TRAVIS_TAG}

README.md

@@ -57,7 +57,7 @@ The source code can be found at https://github.com/kiwix/kiwix-js
 
 ## Unit tests
 
-Unit tests can be run by opening tests.html file on Firefox or Edge (or Chromium/Chrome with some tweaks).
+Unit tests can be run by opening `tests/index.html` file in Firefox, Edge, or Chromium/Chrome.
 
 ## Public releases and nightly builds
 

browser-tests/nightwatch_runner.js

@@ -21,20 +21,6 @@
  */
 'use strict';
 module.exports = {
-    'Unit Tests': function(browser) {
-        // We need to skip the UI tests on Internet Explorer,
-        // because they are not supported, even if the UI tests are supported on this browser
-        if (browser.options.desiredCapabilities.browserName !== "internet explorer") {
-            browser
-                .url('http://localhost:8080/tests.html')
-                .waitForElementVisible('#qunit-testresult', 10000)
-                .pause(10000);
-            browser.expect.element('#qunit-testresult').text.to.contain('tests completed in');
-            browser.expect.element('#qunit-testresult .failed').text.to.equal('0');
-            browser.expect.element('#qunit-testresult .passed').text.not.to.equal('0');
-            browser.end();
-        }
-    },
     'UI Tests': function(browser) {
         browser
             .url('http://localhost:8080/')

package.json

@@ -0,0 +1,23 @@
+{
+  "private": true,
+  "scripts": {
+    "test": "npm run test-unit-local",
+    "test-unit-local": "karma start tests/karma.conf.local.js",
+    "test-unit-saucelabs": "karma start tests/karma.conf.saucelabs.js"
+  },
+  "engines": {
+    "node": ">=10"
+  },
+  "devDependencies": {
+    "http-server": "^0.12.3",
+    "karma": "5.2.3",
+    "karma-chrome-launcher": "3.1.0",
+    "karma-firefox-launcher": "2.1.0",
+    "karma-qunit": "4.1.1",
+    "karma-requirejs": "1.1.0",
+    "karma-sauce-launcher": "^4.3.3",
+    "nightwatch": "^1.5.1",
+    "qunit": "2.12.0",
+    "requirejs": "2.3.3"
+  }
+}

tests.html → tests/index.html

@@ -1,7 +1,8 @@
 <!DOCTYPE html>
 <html>
     <head>
-        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+        <meta charset="utf-8">
+        <base href="../">
         <title>Kiwix-js Unit tests</title>
 
         <!--
@@ -24,23 +25,22 @@
         along with Kiwix (file LICENSE-GPLv3.txt).  If not, see <http://www.gnu.org/licenses/>
         -->
 
-        <link rel="stylesheet" href="tests/qunit-2.3.2.css" />
-        <script src="tests/qunit-2.3.2.js"></script>
+        <link rel="stylesheet" href="node_modules/qunit/qunit/qunit.css" />
+        <script src="node_modules/qunit/qunit/qunit.js"></script>
+        <script>
+            QUnit.config.autostart = false;
+        </script>
 
         <!-- Using require.js, a module system for javascript, include the
              js files. This loads "main.js", which in turn can load other
              files, all handled by require.js:
              http://requirejs.org/docs/api.html#jsfiles -->
-        <script type="text/javascript"
-                data-main="tests/init.js"
-        src="www/js/lib/require.js"></script>
-
-
+        <script src="www/js/lib/require.js" data-main="tests/init.js"></script>
     </head>
     <body>
         <b>NOTE:</b> Firefox and Chrome do not allow access to some local filesystem files used in testing. So, if you're opening this through a file:// URL, you should instead go through a web server : either through a local one (http://localhost/...) or through a remote one (but you need SSL : https://webserver/...).<br/>
         Another option is to force your browser to accept that (but you'll open a security breach) : on Chrome, you can start it with --allow-file-access-from-files command-line argument; on Firefox, you can set privacy.file_unique_origin to false in about:config
         <div id="qunit"></div>
         <div id="qunit-fixture"></div>
     </body>
-</html>
+</html>

tests/init.js

@@ -27,7 +27,7 @@ var params = {};
 var webpMachine = true;
 
 require.config({
-    baseUrl: 'www/js/lib',
+    baseUrl: (window.__karma__ ? 'base/' : '') + 'www/js/lib/',
     paths: {
         'jquery': 'jquery-3.2.1.slim',
         'webpHeroBundle': 'webpHeroBundle_0.0.0-dev.27',

tests/karma.conf.local.js

@@ -0,0 +1,26 @@
+module.exports = function (config) {
+  config.set({
+    basePath: '../',
+    // https://karma-runner.github.io/5.2/config/browsers.html
+    browsers: [
+      'FirefoxHeadless',
+      'ChromeHeadless'
+    ],
+    frameworks: ['qunit'],
+    client: {
+      qunit: {
+        autostart: false
+      }
+    },
+    // logLevel: 'DEBUG',
+    files: [
+      'www/js/lib/xzdec.js',
+      'www/js/lib/require.js',
+      'tests/init.js',
+      { pattern: 'www/**/*', included: false },
+      { pattern: 'tests/**/*', included: false }
+    ],
+    singleRun: true,
+    autoWatch: false
+  });
+};

tests/karma.conf.saucelabs.js

@@ -0,0 +1,84 @@
+module.exports = function (config) {
+  config.set({
+    basePath: '../',
+    // https://karma-runner.github.io/5.2/config/browsers.html
+    // https://github.com/karma-runner/karma-sauce-launcher
+    // https://github.com/karma-runner/karma-sauce-launcher/issues/73
+    sauceLabs: {
+      startConnect: process.env.TRAVIS_JOB_NUMBER ? false : true,
+      username: process.env.SAUCE_USERNAME,
+      accessKey: process.env.SAUCE_ACCESS_KEY,
+      tunnelIdentifier: process.env.TRAVIS_JOB_NUMBER,
+    },
+    customLaunchers: {
+      firefox45: {
+        base: 'SauceLabs',
+        browserName: 'firefox',
+        version: '45.0'
+      },
+      firefox: {
+        base: 'SauceLabs',
+        browserName: 'firefox',
+      },
+      chrome58: {
+        base: 'SauceLabs',
+        browserName: 'chrome',
+        version: '58.0',
+      },
+      chrome: {
+        base: 'SauceLabs',
+        browserName: 'chrome',
+      },
+      edge: {
+        base: 'SauceLabs',
+        browserName: 'MicrosoftEdge',
+      },
+      edge40: {
+        base: 'SauceLabs',
+        browserName: 'MicrosoftEdge',
+        version: '15.15063',
+      },
+      edge44: {
+        base: 'SauceLabs',
+        browserName: 'MicrosoftEdge',
+        version: '18.17763',
+      },
+      ie11: {
+        base: 'SauceLabs',
+        browserName: 'internet explorer',
+      }
+    },
+    // The free account on Sauce does not allow more than 5 concurrent sessions
+    concurrency: 4,
+
+    // REMINDER: Keep this list in sync with the UI tests, in nightwatch.js and .travis.yml.
+    browsers: [
+      // latest edge should be added when Microsoft releases the new Edge based on Chromium
+      'firefox',
+      'chrome',
+      'edge40',
+      'edge44',
+      'firefox45',
+      'chrome58'
+      // Skip unit tests in Internet Explorer due to Promise undefined
+      // 'ie11'
+    ],
+    frameworks: ['qunit'],
+    client: {
+      qunit: {
+        autostart: false
+      }
+    },
+    reporters: ['dots'],
+    logLevel: 'WARN',
+    files: [
+      'www/js/lib/xzdec.js',
+      'www/js/lib/require.js',
+      'tests/init.js',
+      { pattern: 'www/**/*', included: false },
+      { pattern: 'tests/**/*', included: false }
+    ],
+    singleRun: true,
+    autoWatch: false
+  });
+};