Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch 2025.01.1 #2562

Open
wants to merge 54 commits into
base: master
Choose a base branch
from
Open

Patch 2025.01.1 #2562

wants to merge 54 commits into from

Conversation

freearhey
Copy link
Collaborator

@freearhey freearhey commented Jan 1, 2025

Changes:

  • makes the sites/update.test.ts independent of changes to the actual sites folder
  • upgrades eslint config
  • updates channels:lint script
  • fixes all linter issues
  • install husky package to automatically start the lint and channels:lint before commit
  • adds a new check workflow to automatically check modified *.channels.xml and js-files in pull requests
  • install jest-offline package to prevent sending real requests to the server in tests

Test results:

npm test --- commands

> test
> run-script-os commands


> test:default
> TZ=Pacific/Nauru npx jest --runInBand commands

 PASS  tests/commands/epg/grab.test.ts (19.012 s)
 PASS  tests/commands/channels/lint.test.ts (5.77 s)
 PASS  tests/commands/channels/validate.test.ts
 PASS  tests/commands/channels/editor.test.ts
 PASS  tests/commands/channels/parse.test.ts
 PASS  tests/commands/sites/update.test.ts
 PASS  tests/commands/api/generate.test.ts

Test Suites: 7 passed, 7 total
Tests:       16 passed, 16 total
Snapshots:   0 total
Time:        39.833 s, estimated 40 s
Ran all test suites matching /commands/i.

@freearhey freearhey marked this pull request as draft January 1, 2025 07:59
@freearhey freearhey marked this pull request as ready for review January 1, 2025 10:34
BellezaEmporium
BellezaEmporium previously approved these changes Jan 2, 2025
Copy link
Contributor

@BellezaEmporium BellezaEmporium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please note that on certain cases I have a warning that says Node 16 is unmaintained, maybe we should bump it to 20 or 18.

@freearhey
Copy link
Collaborator Author

The project already supports version 22 of node.js.

Try updating your local node, maybe that will help with the warning.

@PopeyeTheSai10r
Copy link
Collaborator

Test Failed
npm run test -- commands

> test
> run-script-os commands


> test:win32
> SET "TZ=Pacific/Nauru" && npx jest --runInBand commands

 FAIL  tests/commands/epg/grab.test.ts (90.331 s)epg:grab  can grab epg by site name

    ENOENT: no such file or directory, open 'C:\Github repositories\epg\tests\__data__\output\guide.xml'

      93 |
      94 | function content(filepath: string) {
    > 95 |   return fs.readFileSync(path.resolve(filepath), {
         |             ^
      96 |     encoding: 'utf8'
      97 |   })
      98 | }

      at content (tests/commands/epg/grab.test.ts:95:13)
      at Object.<anonymous> (tests/commands/epg/grab.test.ts:25:12)

   epg:grab  can grab epg with gzip option enabled

    ENOENT: no such file or directory, open 'C:\Github repositories\epg\tests\__data__\output\guide.xml'

      93 |
      94 | function content(filepath: string) {
    > 95 |   return fs.readFileSync(path.resolve(filepath), {
         |             ^
      96 |     encoding: 'utf8'
      97 |   })
      98 | }

      at content (tests/commands/epg/grab.test.ts:95:13)
      at Object.<anonymous> (tests/commands/epg/grab.test.ts:45:12)

 FAIL  tests/commands/channels/validate.test.ts (22.297 s)
   channels:validate  will show a message if the file contains a duplicate

    expect(received).toContain(expected) // indexOf

    Expected substring: "
    > channels:validate
    > npx tsx scripts/commands/channels/validate.ts --channels=tests/__data__/input/channels-validate/duplicate.channels.xml·
    options:
      channels: tests/__data__/input/channels-validate/duplicate.channels.xml
    tests/__data__/input/channels-validate/duplicate.channels.xml
    ┌─────────┬─────────────┬──────┬────────────────┬─────────┬─────────┐
     (index)  type         lang  xmltv_id        site_id  name    
    ├─────────┼─────────────┼──────┼────────────────┼─────────┼─────────┤
     0        'duplicate'  'en'  'BravoEast.us'  '140'    'Bravo' 
    └─────────┴─────────────┴──────┴────────────────┴─────────┴─────────┘·
    1 error(s) in 1 file(s)
    "
    Received string:    "
    > channels:validate
    > npx tsx scripts/commands/channels/validate.ts --channels=tests/__data__/input/channels-validate/duplicate.channels.xml·
    options:
      channels: tests/__data__/input/channels-validate/duplicate.channels.xml
    tests\\__data__\\input\\channels-validate\\duplicate.channels.xml
    ┌─────────┬─────────────┬──────┬────────────────┬─────────┬─────────┐
     (index)  type         lang  xmltv_id        site_id  name    
    ├─────────┼─────────────┼──────┼────────────────┼─────────┼─────────┤
     0        'duplicate'  'en'  'BravoEast.us'  '140'    'Bravo' 
    └─────────┴─────────────┴──────┴────────────────┴─────────┴─────────┘·
    1 error(s) in 1 file(s)
    "

      20 |     } catch (error) {
      21 |       expect((error as ExecError).status).toBe(1)
    > 22 |       expect((error as ExecError).stdout).toContain(`
         |                                           ^
      23 | > channels:validate
      24 | > npx tsx scripts/commands/channels/validate.ts --channels=tests/__data__/input/channels-validate/duplicate.channels.xml
      25 |

      at Object.<anonymous> (tests/commands/channels/validate.test.ts:22:43)

  ● channels:validate › will show a message if the file contains a channel with wrong xmltv_id

    expect(received).toContain(expected) // indexOf

    Expected substring: "
    > channels:validate
    > npx tsx scripts/commands/channels/validate.ts --channels=tests/__data__/input/channels-validate/wrong_xmltv_id.channels.xml·
    options:
      channels: tests/__data__/input/channels-validate/wrong_xmltv_id.channels.xml
    tests/__data__/input/channels-validate/wrong_xmltv_id.channels.xml
    ┌─────────┬──────────────────┬──────┬────────────────────┬─────────┬─────────────────────┐
    │ (index) │ type             │ lang │ xmltv_id           │ site_id │ name                │
    ├─────────┼──────────────────┼──────┼────────────────────┼─────────┼─────────────────────┤
    │ 0       │ 'wrong_xmltv_id' │ 'en' │ 'CNNInternational' │ '140'   │ 'CNN International' │
    └─────────┴──────────────────┴──────┴────────────────────┴─────────┴─────────────────────┘·
    1 error(s) in 1 file(s)
    "
    Received string:    "
    > channels:validate
    > npx tsx scripts/commands/channels/validate.ts --channels=tests/__data__/input/channels-validate/wrong_xmltv_id.channels.xml·
    options:
      channels: tests/__data__/input/channels-validate/wrong_xmltv_id.channels.xml
    tests\\__data__\\input\\channels-validate\\wrong_xmltv_id.channels.xml
    ┌─────────┬──────────────────┬──────┬────────────────────┬─────────┬─────────────────────┐
    │ (index) │ type             │ lang │ xmltv_id           │ site_id │ name                │
    ├─────────┼──────────────────┼──────┼────────────────────┼─────────┼─────────────────────┤
    │ 0       │ 'wrong_xmltv_id' │ 'en' │ 'CNNInternational' │ '140'   │ 'CNN International' │
    └─────────┴──────────────────┴──────┴────────────────────┴─────────┴─────────────────────┘·
    1 error(s) in 1 file(s)
    "

      45 |     } catch (error) {
      46 |       expect((error as ExecError).status).toBe(1)
    > 47 |       expect((error as ExecError).stdout).toContain(`
         |                                           ^
      48 | > channels:validate
      49 | > npx tsx scripts/commands/channels/validate.ts --channels=tests/__data__/input/channels-validate/wrong_xmltv_id.channels.xml
      50 |

      at Object.<anonymous> (tests/commands/channels/validate.test.ts:47:43)

 FAIL  tests/commands/channels/editor.test.ts (18.518 s)
   channels:editor  shows list of options for a channel

    expect(received).toContain(expected) // indexOf

    Expected substring: "CNN International Europe | CNNInternationalEurope.us [api]"
    Received string:    "
    > channels:editor
    > npx tsx scripts/commands/channels/editor.ts tests/__data__/output/channels.xml·
    ? Choose xmltv_id for \"CNN International\" (140): (Use arrow keys)
    > CNN International | CNNInternational.us [new]·
      CNN International Europe | CNNInternationalEurope.us·
      Overwrite·
      Skip
    File 'tests/__data__/output/channels.xml' successfully saved
    "

      30 |       expect((error as ExecError).status).toBe(1)
      31 |       expect((error as ExecError).stdout).toContain('CNN International | CNNInternational.us [new]')
    > 32 |       expect((error as ExecError).stdout).toContain(
         |                                           ^
      33 |         'CNN International Europe | CNNInternationalEurope.us [api]'
      34 |       )
      35 |       expect((error as ExecError).stdout).toContain('Overwrite')

      at Object.<anonymous> (tests/commands/channels/editor.test.ts:32:43)

'DOT_SITES_DIR' is not recognized as an internal or external command,
operable program or batch file.
 FAIL  tests/commands/sites/update.test.ts
   can update SITES.md

    Command failed: DOT_SITES_DIR=tests/__data__/output/.sites SITES_DIR=tests/__data__/input/sites-update/sites npm run sites:update
    'DOT_SITES_DIR' is not recognized as an internal or external command,
    operable program or batch file.

      17 |
      18 | it('can update SITES.md', () => {
    > 19 |   execSync(
         |           ^
      20 |     'DOT_SITES_DIR=tests/__data__/output/.sites SITES_DIR=tests/__data__/input/sites-update/sites npm run sites:update',
      21 |     {
      22 |       encoding: 'utf8'

      at Object.<anonymous> (tests/commands/sites/update.test.ts:19:11)


node:internal/process/promises:394
    triggerUncaughtException(err, true /* fromPromise */);
    ^
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:217:11)
    at defaultLoad (node:internal/modules/esm/load:109:3)
    at nextLoad (node:internal/modules/esm/hooks:748:28)
    at load (file:///C:/Github%20repositories/epg/node_modules/tsx/dist/esm/index.mjs?1735856181618:2:1768)
    at nextLoad (node:internal/modules/esm/hooks:748:28)
    at Hooks.load (node:internal/modules/esm/hooks:385:26)
    at handleMessage (node:internal/modules/esm/worker:199:24)
    at Immediate.checkForMessages (node:internal/modules/esm/worker:141:28)
    at process.processImmediate (node:internal/timers:491:21) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

Node.js v22.12.0
 FAIL  tests/commands/channels/parse.test.ts (10.949 s)channels:parse  can parse channels

    Command failed: npm run channels:parse --- --config=tests/__data__/input/channels-parse/channels-parse.config.js --output=tests/__data__/output/channels.xml

    node:internal/process/promises:394
        triggerUncaughtException(err, true /* fromPromise */);
        ^
    Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'

      at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:217:11)
      at defaultLoad (node:internal/modules/esm/load:109:3)
      at nextLoad (node:internal/modules/esm/hooks:748:28)
      at load (file:/C:/Github%20repositories/epg/node_modules/tsx/dist/esm/index.mjs?1735856181618:2:1768)
      at nextLoad (node:internal/modules/esm/hooks:748:28)
      at Hooks.load (node:internal/modules/esm/hooks:385:26)
      at handleMessage (node:internal/modules/esm/worker:199:24)
      at Immediate.checkForMessages (node:internal/modules/esm/worker:141:28)
      at process.processImmediate (node:internal/timers:491:21) {
        code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
      }

      Node.js v22.12.0
      at Object.<anonymous> (tests/commands/channels/parse.test.ts:17:13)

 PASS  tests/commands/api/generate.test.ts (11.501 s)
node:internal/modules/cjs/loader:1734
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: The module '\\?\C:\Github repositories\epg\node_modules\libxmljs2\build\Release\xmljs.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 108. This version of Node.js requires
NODE_MODULE_VERSION 127. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`).
    at Object..node (node:internal/modules/cjs/loader:1734:18)
    at Module.load (node:internal/modules/cjs/loader:1318:32)
    at Function._load (node:internal/modules/cjs/loader:1128:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:219:24)
    at Module.require (node:internal/modules/cjs/loader:1340:12)
    at require (node:internal/modules/helpers:138:16)
    at bindings (C:\Github repositories\epg\node_modules\bindings\bindings.js:112:48)
    at Object.<anonymous> (C:\Github repositories\epg\node_modules\libxmljs2\lib\bindings.js:1:37)
    at Module._compile (node:internal/modules/cjs/loader:1565:14) {
  code: 'ERR_DLOPEN_FAILED'
}

Node.js v22.12.0
 FAIL  tests/commands/channels/lint.test.ts (8.775 s)channels:lint  will show a message if the file contains a syntax error

    expect(received).toContain(expected) // indexOf

    Expected substring: "channels-lint.channels.xml
     3:0  Element 'channel': The attribute 'lang' is required but missing.·
    1 error(s)
    "
    Received string:    "
    > channels:lint
    > npx tsx scripts/commands/channels/lint.ts --channels=tests/__data__/input/channels-lint/channels-lint.channels.xml·
    "

      15 |     } catch (error) {
      16 |       expect((error as ExecError).status).toBe(1)
    > 17 |       expect((error as ExecError).stdout).toContain(
         |                                           ^
      18 |         "channels-lint.channels.xml\n 3:0  Element 'channel': The attribute 'lang' is required but missing.\n\n1 error(s)\n"
      19 |       )
      20 |     }

      at Object.<anonymous> (tests/commands/channels/lint.test.ts:17:43)

Test Suites: 6 failed, 1 passed, 7 total
Tests:       8 failed, 6 passed, 14 total
Snapshots:   0 total
Time:        163.994 s
Ran all test suites matching /commands/i.

@PopeyeTheSai10r PopeyeTheSai10r requested a review from tohenk January 3, 2025 00:14
@freearhey freearhey marked this pull request as draft January 3, 2025 08:42
@PopeyeTheSai10r
Copy link
Collaborator

Some errors, but less than before
FAIL  tests/commands/epg/grab.test.ts (91.258 s)
...
PASS  tests/commands/channels/validate.test.ts (19.383 s)
PASS  tests/commands/channels/editor.test.ts (17.061 s)
PASS  tests/commands/channels/parse.test.ts (14.391 s)
FAIL  tests/commands/sites/update.test.ts (12.025 s)
...
PASS  tests/commands/channels/lint.test.ts (28.503 s)
PASS  tests/commands/api/generate.test.ts (11.055 s)

Test Suites: 2 failed, 5 passed, 7 total
Tests:       3 failed, 13 passed, 16 total
Snapshots:   0 total
Time:        194.657 s, estimated 197 s
Ran all test suites matching /commands/i.
DEBUG=true
SET "DEBUG=true" && npm test --- commands

> test
> run-script-os commands


> test:win32
> SET "TZ=Pacific/Nauru" && npx jest --runInBand commands

 FAIL  tests/commands/epg/grab.test.ts (148.842 s)
   Console

    console.log

      > grab
      > npx tsx scripts/commands/epg/grab.ts --site=example.com --output=C:\Github repositories\epg\tests\__data__\output\guide.xml

      starting...
      config:
        output: C:\Github
        maxConnections: 1
        gzip: false
        site: example.com
      loading channels...
        found 3 channel(s)
      run #1:
        [1/6] example.com (en) - Channel1.us - Oct 19, 2022 (1 programs)
        [2/6] example.com (en) - Channel1.us - Oct 20, 2022 (1 programs)
        [3/6] example.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        [4/6] example.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        [5/6] example.com (en) - Channel2.us - Oct 20, 2022 (0 programs)
        [6/6] example.com (en) - Channel2.us - Oct 19, 2022 (0 programs)
        saving to "C:\Github"...
        done in 00h 00m 00s

      at Object.<anonymous> (tests/commands/epg/grab.test.ts:25:47)

    console.log

      > grab
      > npx tsx scripts/commands/epg/grab.ts --channels=tests/__data__/input/epg-grab/sites/**/*.channels.xml --output=tests/__data__/output/guide.xml

      starting...
      config:
        output: tests/__data__/output/guide.xml
        maxConnections: 1
        gzip: false
        channels: tests/__data__/input/epg-grab/sites/**/*.channels.xml
      loading channels...
        found 6 channel(s)
      run #1:
        [1/12] example.com (en) - Channel1.us - Oct 19, 2022 (1 programs)
        [2/12] example.com (en) - Channel1.us - Oct 20, 2022 (1 programs)
        [3/12] example.com (en) - Channel2.us - Oct 20, 2022 (0 programs)
        [4/12] example2.com (en) - Channel3.us - Oct 20, 2022 (1 programs)
        [5/12] example2.com (en) - Channel4.us - Oct 20, 2022 (1 programs)
        [6/12] example2.com (en) - Channel4.us - Oct 19, 2022 (1 programs)
        [7/12] example2.com (en) - Channel3.us - Oct 19, 2022 (1 programs)
        [8/12] example.com (en) - Channel2.us - Oct 19, 2022 (0 programs)
        [9/12] example.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        [10/12] example.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        [11/12] example2.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        [12/12] example2.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        saving to "tests/__data__/output/guide.xml"...
        done in 00h 00m 00s

      at Object.<anonymous> (tests/commands/epg/grab.test.ts:35:47)

    console.log

      > grab
      > npx tsx scripts/commands/epg/grab.ts --channels=tests/__data__/input/epg-grab/sites/**/*.channels.xml --output=C:\Github repositories\epg\tests\__data__\output\guide.xml --gzip

      starting...
      config:
        output: C:\Github
        maxConnections: 1
        gzip: true
        channels: tests/__data__/input/epg-grab/sites/**/*.channels.xml
      loading channels...
        found 6 channel(s)
      run #1:
        [1/12] example.com (en) - Channel1.us - Oct 19, 2022 (1 programs)
        [2/12] example.com (en) - Channel1.us - Oct 20, 2022 (1 programs)
        [3/12] example.com (en) - Channel2.us - Oct 20, 2022 (0 programs)
        [4/12] example.com (en) - Channel2.us - Oct 19, 2022 (0 programs)
        [5/12] example2.com (en) - Channel4.us - Oct 20, 2022 (1 programs)
        [6/12] example2.com (en) - Channel4.us - Oct 19, 2022 (1 programs)
        [7/12] example2.com (en) - Channel3.us - Oct 20, 2022 (1 programs)
        [8/12] example2.com (en) - Channel3.us - Oct 19, 2022 (1 programs)
        [9/12] example.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        [10/12] example.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        [11/12] example2.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        [12/12] example2.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        saving to "C:\Github"...
        saving to "C:\Github.gz"...
        done in 00h 00m 00s

      at Object.<anonymous> (tests/commands/epg/grab.test.ts:47:47)

    console.log

      > grab
      > npx tsx scripts/commands/epg/grab.ts --channels=tests/__data__/input/epg-grab/sites/example.com/example.com.channels.xml --output=tests/__data__/output/guides/{lang}/{site}.xml

      starting...
      config:
        output: tests/__data__/output/guides/{lang}/{site}.xml
        maxConnections: 1
        gzip: false
        channels: tests/__data__/input/epg-grab/sites/example.com/example.com.channels.xml
      loading channels...
        found 3 channel(s)
      run #1:
        [1/6] example.com (en) - Channel1.us - Oct 19, 2022 (1 programs)
        [2/6] example.com (en) - Channel1.us - Oct 20, 2022 (1 programs)
        [3/6] example.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        [4/6] example.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        [5/6] example.com (en) - Channel2.us - Oct 20, 2022 (0 programs)
        [6/6] example.com (en) - Channel2.us - Oct 19, 2022 (0 programs)
        saving to "tests/__data__/output/guides/en/example.com.xml"...
        saving to "tests/__data__/output/guides/fr/example.com.xml"...
        done in 00h 00m 00s

      at Object.<anonymous> (tests/commands/epg/grab.test.ts:62:47)

    console.log

      > grab
      > npx tsx scripts/commands/epg/grab.ts --channels=tests/__data__/input/epg-grab/sites/example.com/example.com.channels.xml --output=tests/__data__/output/guides/{lang}/{site}.xml --lang=fr

      starting...
      config:
        output: tests/__data__/output/guides/{lang}/{site}.xml
        maxConnections: 1
        gzip: false
        channels: tests/__data__/input/epg-grab/sites/example.com/example.com.channels.xml
        lang: fr
      loading channels...
        found 1 channel(s)
      run #1:
        [1/2] example.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        [2/2] example.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        saving to "tests/__data__/output/guides/fr/example.com.xml"...
        done in 00h 00m 00s

      at Object.<anonymous> (tests/commands/epg/grab.test.ts:76:47)

    console.log

      > grab
      > npx tsx scripts/commands/epg/grab.ts --channels=tests/__data__/input/epg-grab/custom.channels.xml --output=tests/__data__/output/guide.xml

      starting...
      config:
        output: tests/__data__/output/guide.xml
        maxConnections: 1
        gzip: false
        channels: tests/__data__/input/epg-grab/custom.channels.xml
      loading channels...
        found 6 channel(s)
      run #1:
        [1/12] example.com (en) - Channel1.us - Oct 19, 2022 (1 programs)
        [2/12] example.com (en) - Channel1.us - Oct 20, 2022 (1 programs)
        [3/12] example.com (en) - Channel2.us - Oct 19, 2022 (0 programs)
        [4/12] example2.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        [5/12] example2.com (en) - Channel4.us - Oct 20, 2022 (1 programs)
        [6/12] example2.com (en) - Channel4.us - Oct 19, 2022 (1 programs)
        [7/12] example2.com (en) - Channel3.us - Oct 20, 2022 (1 programs)
        [8/12] example2.com (en) - Channel3.us - Oct 19, 2022 (1 programs)
        [9/12] example.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        [10/12] example.com (fr) - Channel1.us - Oct 19, 2022 (1 programs)
        [11/12] example.com (en) - Channel2.us - Oct 20, 2022 (0 programs)
        [12/12] example2.com (fr) - Channel1.us - Oct 20, 2022 (1 programs)
        saving to "tests/__data__/output/guide.xml"...
        done in 00h 00m 00s

      at Object.<anonymous> (tests/commands/epg/grab.test.ts:86:47)

    console.log

      > grab
      > npx tsx scripts/commands/epg/grab.ts --channels=tests/__data__/input/epg-grab/custom.channels.xml --output=tests/__data__/output/guide.xml --timeout=0

      starting...
      config:
        output: tests/__data__/output/guide.xml
        maxConnections: 1
        gzip: false
        channels: tests/__data__/input/epg-grab/custom.channels.xml
        timeout: 0
      loading channels...
        found 6 channel(s)
      run #1:
        [1/12] example.com (en) - Channel1.us - Oct 19, 2022 (0 programs)
          ERR: Connection timeout
        [2/12] example.com (en) - Channel1.us - Oct 20, 2022 (0 programs)
          ERR: Connection timeout
        [3/12] example.com (en) - Channel2.us - Oct 20, 2022 (0 programs)
          ERR: Connection timeout
        [4/12] example2.com (en) - Channel3.us - Oct 20, 2022 (0 programs)
          ERR: Connection timeout
        [5/12] example2.com (en) - Channel4.us - Oct 20, 2022 (0 programs)
          ERR: Connection timeout
        [6/12] example2.com (en) - Channel4.us - Oct 19, 2022 (0 programs)
          ERR: Connection timeout
        [7/12] example2.com (en) - Channel3.us - Oct 19, 2022 (0 programs)
          ERR: Connection timeout
        [8/12] example.com (en) - Channel2.us - Oct 19, 2022 (0 programs)
          ERR: Connection timeout
        [9/12] example.com (fr) - Channel1.us - Oct 20, 2022 (0 programs)
          ERR: Connection timeout
        [10/12] example.com (fr) - Channel1.us - Oct 19, 2022 (0 programs)
          ERR: Connection timeout
        [11/12] example2.com (fr) - Channel1.us - Oct 20, 2022 (0 programs)
          ERR: Connection timeout
        [12/12] example2.com (fr) - Channel1.us - Oct 19, 2022 (0 programs)
          ERR: Connection timeout
        done in 00h 00m 00s

      at Object.<anonymous> (tests/commands/epg/grab.test.ts:96:47)epg:grab  can grab epg by site name

    ENOENT: no such file or directory, open 'C:\Github repositories\epg\tests\__data__\output\guide.xml'

      101 |
      102 | function content(filepath: string) {
    > 103 |   return fs.readFileSync(pathToFileURL(filepath), {
          |             ^
      104 |     encoding: 'utf8'
      105 |   })
      106 | }

      at content (tests/commands/epg/grab.test.ts:103:13)
      at Object.<anonymous> (tests/commands/epg/grab.test.ts:27:12)

   epg:grab  can grab epg with gzip option enabled

    ENOENT: no such file or directory, open 'C:\Github repositories\epg\tests\__data__\output\guide.xml'

      101 |
      102 | function content(filepath: string) {
    > 103 |   return fs.readFileSync(pathToFileURL(filepath), {
          |             ^
      104 |     encoding: 'utf8'
      105 |   })
      106 | }

      at content (tests/commands/epg/grab.test.ts:103:13)
      at Object.<anonymous> (tests/commands/epg/grab.test.ts:49:12)

 FAIL  tests/commands/sites/update.test.ts (13.368 s)
   Console

    console.log

      > sites:update
      > npx tsx scripts/commands/sites/update.ts

      at Object.<anonymous> (tests/commands/sites/update.test.ts:28:45)

   can update SITES.md

    expect(received).toEqual(expected) // deep equality

    Expected: "\"# Sites\\r\\n\\r\\n<!-- prettier-ignore -->\\r\\n<table>\\r\\n  <thead>\\r\\n    <tr><th align=\\\"left\\\">Site</th><th align=\\\"left\\\">Status</th><th align=\\\"left\\\">Notes</th></tr>\\r\\n  </thead>\\r\\n  <tbody>\\r\\n    <tr><td><a href=\\\"sites/iltalehti.fi\\\">iltalehti.fi</a></td><td>🟡</td><td>https://github.com/iptv-or g/epg/issues/2396</td></tr>\\r\\n    <tr><td><a href=\\\"sites/indihometv.com\\\">indihometv.com</a></td><td>🟢</td><td></td></tr>\\ r\\n    <tr><td><a href=\\\"sites/kan.org.il\\\">kan.org.il</a></td><td>🔴</td><td>https://github.com/iptv-org/epg/issues/2273</td></tr>\\r\\n  </tbody>\\r\\n</table>\\r\\n\""
    Received: "\"# Sites\\r\\n\\r\\n<!-- prettier-ignore -->\\r\\n<table>\\n  <thead>\\n    <tr><th align=\\\"left\\\">Site</th><th align=\\\"left\\\">Status</th><th align=\\\"left\\\">Notes</th></tr>\\n  </thead>\\n  <tbody>\\n    <tr><td><a href=\\\"sites/iltalehti.fi\\\">iltalehti.fi</a></td><td>🟡</td><td>https://github.com/iptv-org/epg/issues/23 96</td></tr>\\n    <tr><td><a href=\\\"sites/indihometv.com\\\">indihometv.com</a></td><td>🟢</td><td></td></tr>\\n    <tr><td ><a href=\\\"sites/kan.org.il\\\">kan.org.il</a></td><td>🔴</td><td>https://github.com/iptv-org/epg/issues/2273</td></tr>\\n  </tbody>\\n</table>\\r\\n\""

      28 |   if (process.env.DEBUG === 'true') console.log(stdout)
      29 |
    > 30 |   expect(content('tests/__data__/output/sites.md')).toEqual(
         |                                                     ^
      31 |     content('tests/__data__/expected/_sites.md')
      32 |   )
      33 |

      at Object.<anonymous> (tests/commands/sites/update.test.ts:30:53)

 PASS  tests/commands/channels/lint.test.ts (32.282 s)
 PASS  tests/commands/channels/validate.test.ts (22.893 s)
 PASS  tests/commands/channels/editor.test.ts (19.085 s)
 PASS  tests/commands/channels/parse.test.ts (12.597 s)
   Console

    console.log

      > channels:parse
      > npx tsx scripts/commands/channels/parse.ts --config=tests/__data__/input/channels-parse/channels-parse.config.js --output=tests/__data__/output/channels.xml

      File 'tests/__data__/output/channels.xml' successfully saved

      at Object.<anonymous> (tests/commands/channels/parse.test.ts:18:47)

 PASS  tests/commands/api/generate.test.ts (12.141 s)
   Console

    console.log

      > api:generate
      > npx tsx scripts/commands/api/generate.ts

      staring...
      loading channels...
        found 6 channel(s)
      saved to "tests\__data__\output\guides.json"

      at Object.<anonymous> (tests/commands/api/generate.test.ts:20:47)


Test Suites: 2 failed, 5 passed, 7 total
Tests:       3 failed, 13 passed, 16 total
Snapshots:   0 total
Time:        262.374 s
Ran all test suites matching /commands/i.

@BellezaEmporium
Copy link
Contributor

can update SITES.md

expect(received).toEqual(expected) // deep equality

Expected: "\"# Sites\\r\\n\\r\\n<!-- prettier-ignore -->\\r\\n<table>\\r\\n  <thead>\\r\\n    <tr><th align=\\\"left\\\">Site</th><th align=\\\"left\\\">Status</th><th align=\\\"left\\\">Notes</th></tr>\\r\\n  </thead>\\r\\n  <tbody>\\r\\n    <tr><td><a href=\\\"sites/iltalehti.fi\\\">iltalehti.fi</a></td><td>🟡</td><td>https://github.com/iptv-or g/epg/issues/2396</td></tr>\\r\\n    <tr><td><a href=\\\"sites/indihometv.com\\\">indihometv.com</a></td><td>🟢</td><td></td></tr>\\ r\\n    <tr><td><a href=\\\"sites/kan.org.il\\\">kan.org.il</a></td><td>🔴</td><td>https://github.com/iptv-org/epg/issues/2273</td></tr>\\r\\n  </tbody>\\r\\n</table>\\r\\n\""
Received: "\"# Sites\\r\\n\\r\\n<!-- prettier-ignore -->\\r\\n<table>\\n  <thead>\\n    <tr><th align=\\\"left\\\">Site</th><th align=\\\"left\\\">Status</th><th align=\\\"left\\\">Notes</th></tr>\\n  </thead>\\n  <tbody>\\n    <tr><td><a href=\\\"sites/iltalehti.fi\\\">iltalehti.fi</a></td><td>🟡</td><td>https://github.com/iptv-org/epg/issues/23 96</td></tr>\\n    <tr><td><a href=\\\"sites/indihometv.com\\\">indihometv.com</a></td><td>🟢</td><td></td></tr>\\n    <tr><td ><a href=\\\"sites/kan.org.il\\\">kan.org.il</a></td><td>🔴</td><td>https://github.com/iptv-org/epg/issues/2273</td></tr>\\n  </tbody>\\n</table>\\r\\n\""

  28 |   if (process.env.DEBUG === 'true') console.log(stdout)
  29 |
> 30 |   expect(content('tests/__data__/output/sites.md')).toEqual(
     |                                                     ^
  31 |     content('tests/__data__/expected/_sites.md')
  32 |   )
  33 |

  at Object.<anonymous> (tests/commands/sites/update.test.ts:30:53)

Seems that it doesn't return the return lines.

The rest are just files not being located by the program... At first, I would have suspected that the space in the folder name would have triggered Node, but it would have been too easy. If you did an 1:1 copy of the repository, this shouldn't be an issue...

@freearhey freearhey marked this pull request as draft January 6, 2025 04:12
@freearhey
Copy link
Collaborator Author

Fixed (at least I hope so).

Test result from Windows:

C:\Users\freea\test folder\epg>npm test --- commands

> test
> run-script-os commands

> test:win32
> SET "TZ=Pacific/Nauru" && npx jest --runInBand commands

 PASS  tests/commands/epg/grab.test.ts (49.29 s)
 PASS  tests/commands/channels/lint.test.ts (15.521 s)
 PASS  tests/commands/channels/validate.test.ts (11.52 s)
 PASS  tests/commands/channels/editor.test.ts (8.916 s)
 PASS  tests/commands/sites/update.test.ts (6.385 s)
 PASS  tests/commands/channels/parse.test.ts (5.638 s)
 PASS  tests/commands/api/generate.test.ts (6.093 s)

Test Suites: 7 passed, 7 total
Tests:       16 passed, 16 total
Snapshots:   0 total
Time:        104.171 s, estimated 120 s
Ran all test suites matching /commands/i.

@freearhey freearhey marked this pull request as ready for review January 6, 2025 08:40
PopeyeTheSai10r
PopeyeTheSai10r previously approved these changes Jan 6, 2025
@PopeyeTheSai10r
Copy link
Collaborator

I think we are nearly there, but I am seeing some mixed results from my testing.

Here is the output when it "fails". The issue is that tests/commands/epg/grab.test.ts times out.

npm test --- commands

> test
> run-script-os commands


> test:win32
> SET "TZ=Pacific/Nauru" && npx jest --runInBand commands

 FAIL  tests/commands/epg/grab.test.ts (91.363 s)epg:grab  can grab epg with gzip option enabled

    thrown: "Exceeded timeout of 10000 ms for a test.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      40 |   })
      41 |
    > 42 |   it('can grab epg with gzip option enabled', async () => {
         |   ^
      43 |     const cmd = `${ENV_VAR} npm run grab --- --channels=tests/__data__/input/epg-grab/sites/**/*.channels.xml --output="${path.resolve(
      44 |       'tests/__data__/output/guide.xml'
      45 |     )}" --gzip`

      at tests/commands/epg/grab.test.ts:42:3
      at Object.<anonymous> (tests/commands/epg/grab.test.ts:19:1)

 PASS  tests/commands/channels/lint.test.ts (27.789 s)
 PASS  tests/commands/channels/validate.test.ts (21.956 s)
 PASS  tests/commands/channels/editor.test.ts (15.584 s)
 PASS  tests/commands/api/generate.test.ts (10.848 s)
 PASS  tests/commands/sites/update.test.ts (11.22 s)
 PASS  tests/commands/channels/parse.test.ts (11.266 s)

Test Suites: 1 failed, 6 passed, 7 total
Tests:       1 failed, 15 passed, 16 total
Snapshots:   0 total
Time:        191.107 s
Ran all test suites matching /commands/i.

@freearhey freearhey marked this pull request as draft January 6, 2025 18:09
@BellezaEmporium
Copy link
Contributor

Do you have a "punycode" warning when trying to run Jest on Node 22 ? Apparently, a few people do, and Jest wasn't updated to reflect those changes...

I personally am tinkering with Vitest, which is another testing framework, that's quite close to what Jest can provide.

(and before you ask, no, you don't need Vite to play with Vitest.)

If you want some comparison points, I suggest reading this article : https://www.speakeasy.com/post/vitest-vs-jest

@BellezaEmporium
Copy link
Contributor

Oh well... nevermind. The VSCode plugin straight ignores the timezone. All good.

@freearhey
Copy link
Collaborator Author

So, I tried to speed up the test at least to keep them within the timeout on Windows, but it didn't work.

In my case they run faster than for @PopeyeTheSai10r, but still 3 times slower than usual.

macOS
npm test --- grab    

> test
> run-script-os grab


> test:default
> TZ=Pacific/Nauru npx jest --runInBand grab

 PASS  tests/commands/epg/grab.test.ts (15.094 s)
  epg:grab
    ✓ can grab epg by site name (2267 ms)
    ✓ can grab epg with multiple channels.xml files (2073 ms)
    ✓ can grab epg with gzip option enabled (2119 ms)
    ✓ can grab epg with wildcard as output (2050 ms)
    ✓ can grab epg then language filter enabled (2038 ms)
    ✓ can grab epg using custom channels list (2065 ms)
    ✓ it will raise an error if the timeout is exceeded (2049 ms)

Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
Snapshots:   0 total
Time:        15.265 s, estimated 16 s
Ran all test suites matching /grab/i.
Windows 10 VM
npm test --- grab    

> test
> run-script-os grab


> test:win32
> SET "TZ=Pacific/Nauru" && npx jest --runInBand grab

 PASS  tests/commands/epg/grab.test.ts (41.389 s)
  epg:grab
    √ can grab epg by site name (6507 ms)
    √ can grab epg with multiple channels.xml files (5922 ms)
    √ can grab epg with gzip option enabled (5554 ms)
    √ can grab epg with wildcard as output (5389 ms)
    √ can grab epg then language filter enabled (5209 ms)
    √ can grab epg using custom channels list (5458 ms)
    √ it will raise an error if the timeout is exceeded (5886 ms)

Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
Snapshots:   0 total
Time:        41.695 s
Ran all test suites matching /grab/i.

Results: 15.265 s vs 41.695 s

Even if you only run one test:

macOS (single test)
npm test --- grab                                           

> test
> run-script-os grab


> test:default
> TZ=Pacific/Nauru npx jest --runInBand grab

 PASS  tests/commands/epg/grab.test.ts
  epg:grab
    ✓ can grab epg with gzip option enabled (2460 ms)
    ○ skipped can grab epg by site name
    ○ skipped can grab epg with multiple channels.xml files
    ○ skipped can grab epg with wildcard as output
    ○ skipped can grab epg then language filter enabled
    ○ skipped can grab epg using custom channels list
    ○ skipped it will raise an error if the timeout is exceeded

Test Suites: 1 passed, 1 total
Tests:       6 skipped, 1 passed, 7 total
Snapshots:   0 total
Time:        3.174 s, estimated 16 s
Ran all test suites matching /grab/i.
Windows 10 VM (single test)
npm test -- grab

> test
> run-script-os grab


> test:win32
> SET "TZ=Pacific/Nauru" && npx jest --runInBand grab

 PASS  tests/commands/epg/grab.test.ts (6.588 s)
  epg:grab
    √ can grab epg by site name (5505 ms)
    ○ skipped can grab epg with multiple channels.xml files
    ○ skipped can grab epg with gzip option enabled
    ○ skipped can grab epg with wildcard as output
    ○ skipped can grab epg then language filter enabled
    ○ skipped can grab epg using custom channels list
    ○ skipped it will raise an error if the timeout is exceeded

Test Suites: 1 passed, 1 total
Tests:       6 skipped, 1 passed, 7 total
Snapshots:   0 total
Time:        6.772 s, estimated 42 s
Ran all test suites matching /grab/i.

Results: 3.174 s vs 6.772 s

Perhaps the problem is in Jest (jestjs/jest#7631). But with vitest for example the results were even worse:

Windows 10 VM (vitest)
npm test -- grab

> test
> run-script-os grab


> test:win32
> SET "TZ=Pacific/Nauru" && npx vitest run grab


 RUN  v2.1.8 C:/Users/freea/test folder/epg

 ✓ tests/commands/epg/grab.test.ts (7) 49215ms
   ✓ epg:grab (7) 49214ms
     ✓ can grab epg by site name 9934ms
     ✓ can grab epg with multiple channels.xml files 7790ms
     ✓ can grab epg with gzip option enabled 6006ms
     ✓ can grab epg with wildcard as output 6537ms
     ✓ can grab epg then language filter enabled 6197ms
     ✓ can grab epg using custom channels list 6334ms
     ✓ it will raise an error if the timeout is exceeded 6413ms

 Test Files  1 passed (1)
      Tests  7 passed (7)
   Start at  22:15:15
   Duration  52.00s (transform 693ms, setup 0ms, collect 1.43s, tests 49.21s, environment 0ms, prepare 409ms)

Results: 52.00s

At this point, I just increased the timeout up to 30 seconds for test that failed (c436e75). But if you know a more reliable solution let me know.

@freearhey freearhey marked this pull request as ready for review January 7, 2025 17:36
@BellezaEmporium
Copy link
Contributor

BellezaEmporium commented Jan 7, 2025

So, I tried to speed up the test at least to keep them within the timeout on Windows, but it didn't work.

In my case they run faster than for @PopeyeTheSai10r, but still 3 times slower than usual.
macOS

Windows 10 VM

Results: 15.265 s vs 41.695 s

Even if you only run one test:
macOS (single test)

Windows 10 VM (single test)

Results: 3.174 s vs 6.772 s

Perhaps the problem is in Jest (jestjs/jest#7631). But with vitest for example the results were even worse:
Windows 10 VM (vitest)

Results: 52.00s

At this point, I just increased the timeout up to 30 seconds for test that failed (c436e75). But if you know a more reliable solution let me know.

How does it do if you replace the "--runInBand" option with the "--maxWorkers=50%" option ?

@freearhey
Copy link
Collaborator Author

freearhey commented Jan 7, 2025

How does it do if you replace the "--runInBand" option with the "--maxWorkers=50%" option ?

Result: 38.663 s

Log
>npm test --- grab

> test
> run-script-os grab


> test:win32
> SET "TZ=Pacific/Nauru" && npx jest --maxWorkers=50% grab

 PASS  tests/commands/epg/grab.test.ts (38.492 s)
  epg:grab
    √ can grab epg by site name (5448 ms)
    √ can grab epg with multiple channels.xml files (5227 ms)
    √ can grab epg with gzip option enabled (5562 ms)
    √ can grab epg with wildcard as output (5382 ms)
    √ can grab epg then language filter enabled (5304 ms)
    √ can grab epg using custom channels list (5359 ms)
    √ it will raise an error if the timeout is exceeded (5209 ms)

Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
Snapshots:   0 total
Time:        38.663 s
Ran all test suites matching /grab/i.

@BellezaEmporium
Copy link
Contributor

Still very long... but weirdly faster than the --runInBand option... odd.

@PopeyeTheSai10r
Copy link
Collaborator

All good here after running nmp ci

npm test --- commands

> test
> run-script-os commands


> test:win32
> SET "TZ=Pacific/Nauru" && npx jest --runInBand commands

 PASS  tests/commands/epg/grab.test.ts (80.802 s)
 PASS  tests/commands/channels/lint.test.ts (28.356 s)
 PASS  tests/commands/channels/validate.test.ts (21.409 s)
 PASS  tests/commands/channels/editor.test.ts (17.287 s)
 PASS  tests/commands/channels/parse.test.ts (10.888 s)
 PASS  tests/commands/api/generate.test.ts (10.722 s)
 PASS  tests/commands/sites/update.test.ts (10.841 s)

Test Suites: 7 passed, 7 total
Tests:       16 passed, 16 total
Snapshots:   0 total
Time:        180.784 s, estimated 189 s
Ran all test suites matching /commands/i.

@BellezaEmporium
Copy link
Contributor

BellezaEmporium commented Jan 7, 2025

Holy moly, 180.784 s ! That's a heck of a long time, that extra timeout did help for sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants