Setup broken link checker

[robdodson]

@samthor commented on Wed Nov 25 2020

This could happen as a GitHub action after the site is built and a PR is checked.

(If it's way too noisy of course, we'll revisit this.)

[robdodson]

Moving forward this would be a really nice thing to have.

I found linkinator to be a pretty good npm package at detecting broken links and it supports pointing at markdown files (not just urls).

I think there are existing GH Actions out there which will try to crawl the entire site and you can run them on a cron, but since the site is so large I'm unsure if that would be the ideal approach. My hunch is initially we would want something that checks for broken links in PRs and subsequently we might try to figure out some sort of total site crawl thing.

[samthor]

Some custom code which crawls the entire site and its links takes ~7s on my Mac here, and that's without any attempt at parallelism. It finds the following internal links that 404: (with no concern for #-links)

  '/origintrials/',
  '/docs/extensions/mv3/getstarted/examples/tutorials/get_started_complete.zip',
  '/docs/extensions/mv3/getstarted/examples/tutorials/get_started/manifest.json',
  '/docs/extensions/mv3/getstarted/examples/tutorials/get_started/background.js',
  '/docs/extensions/mv3/getstarted/examples/tutorials/get_started/popup.html',
  '/docs/extensions/mv3/getstarted/examples/tutorials/get_started/images.zip',
  '/docs/extensions/mv3/getstarted/examples/tutorials/get_started/popup.js',
  '/docs/extensions/mv3/getstarted/examples/tutorials/get_started/options.html',
  '/docs/extensions/mv3/getstarted/examples/tutorials/get_started/options.js',
  '/docs/extensions/reference/displaySource/',
  '/docs/extensions/reference/dns/',
  '/docs/extensions/reference/idltest/',
  '/docs/extensions/reference/login/',
  '/docs/extensions/reference/loginScreenStorage/',
  '/docs/extensions/examples/api/contextMenus/global_context_search/locales.js',
  '/docs/extensions/storage/',
  '/docs/extensions/mv3/manifest/host_permissions/',
  '/docs/extensions/examples/howto/sandbox/sandbox.html',
  '/docs/extensions/examples/howto/sandbox/eventpage.html',
  '/docs/extensions/examples/howto/sandbox/eventpage.js',
  '/docs/devtools/docs/extensions-gallery/',
  '/docs/devtools/docs/commandline-api/',
  '/docs/devtools/docs/sample-extensions/',
  '/docs/extensions/mv3/tut_oauth/examples/tutorials/oauth_tutorial_complete.zip',
  '/docs/extensions/mv3/tut_oauth/examples/tutorials/oauth_starter/manifest.json',
  '/docs/extensions/mv3/tut_oauth/examples/tutorials/oauth_starter/background.js',
  '/docs/extensions/mv3/tut_oauth/examples/tutorials/oauth_starter/index.html',
  '/docs/extensions/mv3/tut_oauth/examples/tutorials/oauth_starter/oauth.js',
  '/docs/extensions/inform_users/',
  '/docs/extensions/packaging/',
  '/docs/extensions/npapi/',
  '/packaging/',
  '/docs/extensions/mv2/getstarted/examples/tutorials/get_started/manifest.json',
  '/docs/extensions/mv2/getstarted/examples/tutorials/get_started/background.js',
  '/docs/extensions/mv2/getstarted/examples/tutorials/get_started/popup.html',
  '/docs/extensions/mv2/getstarted/examples/tutorials/get_started/popup.js',
  '/docs/extensions/mv2/getstarted/examples/tutorials/get_started/options.html',
  '/docs/extensions/mv2/getstarted/examples/tutorials/get_started/options.js',
  '/docs/extensions/mv2/tut_oauth/examples/tutorials/oauth_tutorial_complete.zip',
  '/docs/extensions/mv2/tut_oauth/examples/tutorials/oauth_starter/manifest.json',
  '/docs/extensions/mv2/tut_oauth/examples/tutorials/oauth_starter/background.js',
  '/docs/extensions/mv2/tut_oauth/examples/tutorials/oauth_starter/index.html',
  '/docs/extensions/mv2/tut_oauth/examples/tutorials/oauth_starter/oauth.js',
  '/docs/extensions/mv2/richNotifications/inform_users/',
  '/publish/',
  '/docs/apps/npapi/',
  '/docs/webstore/check_for_payment/',
  '/docs/webstore/intl/en/program_policies.html',
  '/docs/apps/about_apps/',
  '/docs/extensions/index/',
  '/docs/extensions/mv2/tabs.html',
  '/static/images/calculator-16.png',
  '/static/images/calculator-128.png',
  '/docs/apps/app_codelab_intro/app_codelab_basics.html',
  '/docs/apps/app_codelab_intro/app_codelab_todomvc.html',
  '/docs/apps/app_codelab_intro/app_codelab_alarms.html',
  '/docs/apps/app_codelab_intro/app_codelab_webview.html',
  '/docs/apps/app_codelab_intro/app_codelab_images.html',
  '/docs/apps/app_codelab_intro/app_codelab_filesystem.html',
  '/docs/apps/app_codelab_intro/app_codelab_publish.html',
  '/docs/apps/app_codelab_basics/app_codelab_alarms.html',
  '/docs/apps/app_codelab_basics/app_lifecycle.html',
  '/docs/apps/app_codelab_basics/app_codelab_import_todomvc.html',
  '/docs/apps/app_codelab_import_todomvc/app_codelab_alarms.html',
  '/docs/apps/app_codelab_alarms/app_codelab_webview.html',
  '/docs/apps/app_codelab_webview/app_codelab_images.html',
  '/docs/apps/app_codelab_images/app_codelab_filesystem.html',
  '/docs/apps/app_codelab_filesystem/app_codelab_publish.html',
  '/chrome/apps/docs/developers_guide/',
  '/docs/apps/nativeMessaging/getstarted/',
  '/docs/apps/app_bluetooth/bluetooth.html',
  '/docs/apps/app_bluetooth/bluetoothSocket.html',
  '/docs/apps/app_bluetooth/bluetoothLowEnergy.html',
  '/docs/apps/angular_framework/identityhtml/',
  '/docs/apps/angular_framework/contentSecurityPolicyhtml/',
  '/docs/apps/angular_framework/app_identityhtml/',
  '/static/images/playcanvas_designer2.png',
  '/docs/apps/autoupdate/packaging/',
  '/docs/apps/autoupdate/npapi/',
  '/web/updates/images/2018/09/notch-extra-margin.png',
  '/docs/extensions/reference/runtime/manifest/externally_connectable.html',
  '/docs/extensions/reference/runtime/manifest.html',
  '/docs/extensions/optionsV2/',
  '/docs/extensions/reference/declarativeContent/declare_permissions.html',
  '/docs/extensions/reference/declarativeContent/activeTab.html',
  '/docs/extensions/cloudMessaging/',
  '/docs/extensions/reference/identity/app_identity.html',
  '/docs/native-client/overview/',
  '/docs/webstore/developer/dashboard/',
  '/docs/webstore/pricing/money.html',
  '/docs/apps/analytics.html',
  '/docs/webstore/index/',
  '/docs/apps/storage.html',
  '/docs/apps/app_runtime.html',
  '/docs/apps/app_window.html',
  '/docs/apps/alarms.html',
  '/docs/apps/tags/webview/',
  '/docs/apps/fileSystem.html',
  '/twitter.com/chromiumdev/',
  '/docs/webstore/payments-iap/webstore/webstore_api/payments/list/',

[robdodson]

'/origintrials/',

hm weird. that doesn't seem to 404 for me?

[samthor]

I was running this against localhost, so origintrials/devsummit aren't available, they're provided via the GFE.

…

On Sat, 30 Jan 2021 at 09:33, Rob Dodson ***@***.***> wrote: '/origintrials/', hm weird. that doesn't seem to 404 for me? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#100 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA5DEAV4N3VPIZZODSZEV3S4MZTBANCNFSM4U63I54Q> .

[matthiasrohmer]

Closing in favor of GoogleChrome/webdev-infra#46.