use webpack 5

[chenxsan]

closes #3961

• clean any unused dependencies if there are
  • clean @mdx-js/mdx from devDependencies since we don't use it directly anywhere
  • clean uglifyjs-webpack-plugin
  • clean handlebars (introduced in chore(deps) upgrade security vulnerable deps, upgrade travis node to … #3429 for security fix but there's no need now)
  • clean ajv (introduced in https://github.com/webpack/webpack.js.org/pull/980/files#diff-b9cfc7f2cdf78a7f4b91a753d10865a2R102)
  • clean whatwg-fetch (introduced in 15e0494 and removed in a5566a1#diff-4343058ca05c3116fb4a825bcc162701)
  • clean through2 (introduced in 7700a5f and the code was removed in https://github.com/webpack/webpack.js.org/pull/3143/files#diff-7a9e4cf384a87153a355502185296781)
  • clean loader-utils (introduced in 458141f and removed in 28c0855#diff-15ae716bc29a8465762f4edbb8856ad2)
• upgrade any outdated dependencies
  • upgrade @mdx-js/loader from 0.15.7 to 1.6.16
  • upgrade file-loader from 1.1.11 to 6.1.0
  • upgrade mini-css-extract-plugin from 0.5.0 to 0.11.1
  • upgrade css-loader from 0.28.11 to 4.3.0
  • upgrade style-loader from 0.18.2 to 1.2.1
  • upgrade postcss-loader from 2.1.6 to 4.0.1
  • upgrade autoprefixer from 7.2.6 to 9.8.6
  • upgrade sass-loader from 6.0.7 to 10.0.2
  • upgrade remark-loader from 0.3.2 to 1.0.0
  • upgrade copy-webpack-plugin from 4.5.2 to 6.1.0
  • upgrade terser-webpack-plugin from 2.3.5 to 4.1.0
  • upgrade mkdirp from 0.5.4 to 1.0.4
  • upgrade @octokit/rest from 16.27.1 to 18.0.5 (TODO: filter out all archived repos)
  • upgrade alex from 5.1.0 to 9.0.1 (would break the CI as the new version produces a lot of warnings)
  • upgrade rimraft from 2.6.2 to 3.0.2
  • upgrade webpack-merge from 4.2.2 to 5.1.4
  • upgrade bundlesize from 0.17.1 to 0.18.0
  • upgrade jest from 23.6.0 to 26.4.2
  • upgrade gh-pages from 1.0.0 to 3.1.0
  • upgrade lint-staged from 8.1.0 to 10.3.0
  • upgrade cypress from 4.5.0 to 5.1.0
  • upgrade husky from 1.0.0-rc.8 to 4.3.0
  • upgrade sirv-cli from 0.1.2 to 1.0.6
  • upgrade markdownlint from 0.11.0 to 0.20.4
  • upgrade markdownlint-cli from 0.13.0 to 0.23.2
  • upgrade directory-tree from 2.2.0 to ^2.2.4
  • upgrade directory-tree-webpack-plugin from 0.3 to 1.0.2
  • upgrade http-server from 0.10.0 to 0.12.3
  • upgrade redirect-webpack-plugin from 0.1.1 to 1.0.0
  • upgrade html-webpack-plugin from 3.2.0 to 4.4.1
  • upgrade react-router-dom from 4.3.1 to 5.2.0
  • upgrade react-tiny-popover from 3.4.2 to 5.1.0
  • upgrade eslint-loader from 2.2.1 to 4.0.2
  • upgrade front-matter from 2.3.0 to 4.0.2
  • upgrade remark-slug from 5.1.2 to 6.0.0
  • upgrade remark-autolink-headings from 5.2.1 to 6.0.1
  • upgrade react-markdown from 4.2.2 to 4.3.1
• remove incompatible dependencies
  • remove html-webpack-template as it's not compatible with html-webpack-plugin v4 which is required for webpack 5
• upgrade webpack to 5
  • terser-webpack-plugin might have memory leak (fixed in the latest version)
  • offline-plugin is incompatible use webpack 5 #3963 (comment) (replaced with workbox-webpack-plugin from google)
  • replace react-hot-loader with react-refresh-webpack-plugin

Make the project fast

So far the development server takes about 12s to start while recompiling takes about 3s. It would be great if we can improve the development speed which saves a lot of time for the contributors.

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/webpack-docs/webpack-js-org/aiv2hark1
✅ Preview: https://webpack-js-org-git-feature-use-webpack5.webpack-docs.vercel.app

[chenxsan]

@mdx-js/[email protected] breaks CI https://travis-ci.org/github/webpack/webpack.js.org/jobs/725226358#L1396:

ERROR in TypeError: Cannot call a class as a function
    at evalmachine.<anonymous>:93:571097
    at s (evalmachine.<anonymous>:93:571148)
    at s.value (evalmachine.<anonymous>:51:131344)
    at o (evalmachine.<anonymous>:44:11784)
    at jn (evalmachine.<anonymous>:44:12058)
    at n.s.render (evalmachine.<anonymous>:44:14574)
    at n.s.read (evalmachine.<anonymous>:44:14150)
    at Object.renderToString (evalmachine.<anonymous>:44:19751)
    at s.default (evalmachine.<anonymous>:36:65754)
    at /home/travis/build/webpack/webpack.js.org/node_modules/static-site-generator-webpack-plugin/index.js:77:23
error Command failed with exit code 2.

Updated

Seems to be a bug according to mdx-js/mdx#443, it's advised to upgrade to v1.

[sokra]

I added a few improvements. I also enabled persistent caching.

So far the development server takes about 12s to start while recompiling takes about 3s. It would be great if we can improve the development speed which saves a lot of time for the contributors.

Ok you have a pretty fast computer. For me it takes 56s initially and 9.6s for recompilation. And 19s for an initial build with persistent cache available.

You can get a performance profile with yarn start --profile.

My results

<w> [webpack.Progress]  |  | 4973 ms watch run > ESLintWebpackPlugin
<w> [webpack.Progress]  | 4988 ms setup > watch run
<w> [webpack.Progress]  |  | 4663 ms compile > DirectoryTreeWebpackPlugin
<w> [webpack.Progress]  | 4676 ms setup > compile
<i> [webpack.Progress]  | 66 ms setup > compilation
<w> [webpack.Progress] 9741 ms setup
<i> [webpack.Progress]  | 29 ms building > import loader ../node_modules/@pmmmwh/react-refresh-webpack-plugin/loader/index.js
<i> [webpack.Progress]  | 70 ms building > import loader ../node_modules/babel-loader/lib/index.js
<w> [webpack.Progress]  | 2283 ms building > entries dependencies modules
<i> [webpack.Progress]  | 359 ms building > import loader ../node_modules/html-loader/dist/cjs.js
<i> [webpack.Progress]  | 32 ms building > import loader ../node_modules/remark-loader/dist/cjs.js
<w> [webpack.Progress]  | 8924 ms building > entries dependencies modules
<i> [webpack.Progress]  | 77 ms building > import loader ../node_modules/mini-css-extract-plugin/dist/loader.js
<i> [webpack.Progress]  | 702 ms building > import loader ../node_modules/css-loader/dist/cjs.js
<i> [webpack.Progress]  | 140 ms building > import loader ../node_modules/postcss-loader/dist/cjs.js
<i> [webpack.Progress]  | 109 ms building > import loader ../node_modules/sass-loader/dist/cjs.js
<i> [webpack.Progress]  | 282 ms building > entries dependencies modules
<i> [webpack.Progress]  | 584 ms building > import loader ../node_modules/fontgen-loader/index.js
<i> [webpack.Progress]  | 249 ms building > entries dependencies modules
<i> [webpack.Progress]  | 288 ms building > import loader ../node_modules/@mdx-js/loader/index.js
<e> [webpack.Progress]  | 24292 ms building > entries dependencies modules
<e> [webpack.Progress] 39258 ms building
<i> [webpack.Progress]  |  | 118 ms finish module graph > FlagDependencyExportsPlugin
<i> [webpack.Progress]  | 168 ms sealing > finish module graph
<i> [webpack.Progress]  | 78 ms sealing > chunk graph
<i> [webpack.Progress]  |  | 23 ms chunk optimization > SplitChunksPlugin
<i> [webpack.Progress]  | 29 ms sealing > chunk optimization
<i> [webpack.Progress]  | 11 ms sealing > chunk ids
<i> [webpack.Progress]  | 120 ms sealing > module hashing
<i> [webpack.Progress]  | 271 ms sealing > code generation
<i> [webpack.Progress]  | 23 ms sealing > runtime requirements
<i> [webpack.Progress]  | 75 ms sealing > hashing
<i> [webpack.Progress]  | 208 ms sealing > chunk assets processing
<w> [webpack.Progress]  |  | 6137 ms asset processing > SourceMapDevToolPlugin
<w> [webpack.Progress]  | 6146 ms sealing > asset processing
<i> [webpack.Progress]  | 16 ms sealing > after asset optimization
<i> [webpack.Progress]  | 15 ms sealing > recording
<w> [webpack.Progress] 7221 ms sealing
<i> [webpack.Progress]  |  | 27 ms emit > HtmlWebpackPlugin
<i> [webpack.Progress]  | 324 ms emitting > emit
<i> [webpack.Progress] 332 ms emitting
<i> [webpack.Progress]  |  | 24 ms plugins > webpack-dev-server
<i> [webpack.Progress]  | 27 ms done > plugins
<i> [webpack.Progress] 29 ms done
webpack 5.0.0-rc.4 compiled successfully in 56554 ms
i ｢wdm｣: Compiled successfully.
i ｢wdm｣: Compiling...
<w> [webpack.Progress]  |  | 2056 ms watch run > ESLintWebpackPlugin
<w> [webpack.Progress]  | 2059 ms setup > watch run
<w> [webpack.Progress]  |  | 4086 ms compile > DirectoryTreeWebpackPlugin
<w> [webpack.Progress]  | 4111 ms setup > compile
<i> [webpack.Progress]  | 38 ms setup > compilation
<w> [webpack.Progress] 6213 ms setup
<i> [webpack.Progress]  | 58 ms building > entries dependencies modules
<i> [webpack.Progress]  | 23 ms building > entries dependencies modules
<w> [webpack.Progress]  | 1820 ms building > entries dependencies modules
<w> [webpack.Progress] 1954 ms building
<i> [webpack.Progress]  |  | 25 ms finish module graph > FlagDependencyExportsPlugin
<i> [webpack.Progress]  | 53 ms sealing > finish module graph
<i> [webpack.Progress]  |  | 11 ms plugins > WarnCaseSensitiveModulesPlugin
<i> [webpack.Progress]  | 15 ms sealing > plugins
<i> [webpack.Progress]  | 25 ms sealing > chunk graph
<i> [webpack.Progress]  |  | 15 ms chunk optimization > SplitChunksPlugin
<i> [webpack.Progress]  | 24 ms sealing > chunk optimization
<i> [webpack.Progress]  | 87 ms sealing > module hashing
<i> [webpack.Progress]  | 28 ms sealing > code generation
<i> [webpack.Progress]  | 15 ms sealing > runtime requirements
<i> [webpack.Progress]  | 56 ms sealing > hashing
<i> [webpack.Progress]  | 78 ms sealing > chunk assets processing
<i> [webpack.Progress]  |  | 958 ms asset processing > SourceMapDevToolPlugin
<i> [webpack.Progress]  | 974 ms sealing > asset processing
<w> [webpack.Progress] 1414 ms sealing
<i> [webpack.Progress]  | 89 ms emitting > emit
<i> [webpack.Progress] 93 ms emitting
<i> [webpack.Progress]  |  | 15 ms plugins > webpack-dev-server
<i> [webpack.Progress]  | 17 ms done > plugins
<i> [webpack.Progress] 19 ms done
webpack 5.0.0-rc.4 compiled successfully in 9676 ms

Most notable:

• ESLintWebpackPlugin, DirectoryTreeWebpackPlugin and SourceMapDevToolPlugin are expensive
  • devtool: 'source-map' is a bad choice for incremental build -> devtool: "eval-source-map" or even devtool: "eval-cheap-module-source-map" would be better.
  • Do we really need to run eslint during the build. Maybe enable lintDirtyModulesOnly

[chenxsan]

Maybe we can remove ESLintWebpackPlugin so users can use whatever they have in their editors to fix those eslint warnings. For example, I would rather use auto formatting in VSCode than waiting ESLintWebpackPlugin to fix code warning as it's very slow compared to VSCode's. Also, we already have those lint jobs running in CI to ensure the code quality.

[sokra]

Maybe we can remove ESLintWebpackPlugin so users can use whatever they have in their editors to fix those eslint warnings. For example, I would rather use auto formatting in VSCode than waiting ESLintWebpackPlugin to fix code warning as it's very slow compared to VSCode's. Also, we already have those lint jobs running in CI to ensure the code quality.

I'll remove the plugin. In long term prettier would be the more solid choice, but that probably changes a lot code and is better as separate PR.

[chenxsan]

There was a PR #3109 to integrate prettier, not sure what's blocking it.

[alexander-akait]

I think always what linting is out of scope bundler and should be setup on CI and staged step 😄

[sokra]

Thanks