Development performance improvements

[vjpr]

After discussions with @timneutkens, here are several ways to make Next.js faster to work with during development.

• hard-source-webpack-plugin
• resolve.unsafeCache
• autodll-webpack-plugin - Share common deps with disk caching.
• Webpack@5 module disk cache ([spec: webpack 5] - A module disk cache between build processes webpack/webpack#6527)
• cache-loader - Caches the result of following loaders on disk
• Prebuild next.js pages on server start
• Allow multiple pages in the same bundle
• Reduce size of common deps
  • Lodash = babel-plugin-lodash + lodash-webpack-plugin + https://medium.com/making-internets/why-using-chain-is-a-mistake-9bc1f80d51ba

Metrics

• Server start time.
• Incremental build time for changes on same page (HMR).
• Direct page visit (typing in url).
• Navigating to new page via a link.
• SSR of React Component

Profiling tools

• Chrome DevTools --inspect - profile any part of the app (including flamecharts with a slightly easier workflow than ProfilerPlugin I think)
• ProfilerPlugin in Webpack@4 - generates flamecharts for builds
• webpackDevMiddleware.stats = 'verbose'
• Monkey-patch next build to show stats.toString()
• WebpackAnalyzePlugin
• require('perf_hooks') - https://nodejs.org/api/perf_hooks.html

[vjpr]

autodll-webpack-plugin

I tried this and it works quite well out of the box. The website says it will become obsolete soon because of hard-source and webpack@5, but its still quite useful today, especially for sharing deps between Next.js page bundles.

[timneutkens]

autodll-webpack-plugin has been implemented for react and react-dom at least 👍

[timneutkens]

resolve.unsafeCache is true by default apparently, so there's no gain there, unfortunately.

[timneutkens]

Implemented hard-source-webpack-plugin in #5181. Releasing it on canary right now.

[vjpr]

@timneutkens We should try webpack 5 with built-in disk cache https://twitter.com/TheLarkInn/status/1069246795031576576

[timneutkens]

@divmain is working on build optimizations, starting with this PR: #5810

[bz-stripe]

Have there been any updates on this work? Especially #4843

[kachkaev]

Possibly related issue (still needs investigation): #9153

[timneutkens]

We're investigating other optimizations currently.

[glopezep]

@timneutkens is it will be possible Allow multiple pages in the same bundle for development performance improvement #4843 ? because navigating is really slowly but only on development

[glopezep]

Does someone know what happened to this?

After discussions with @timneutkens, here are several ways to make Next.js faster to work with during development.

• hard-source-webpack-plugin

• resolve.unsafeCache

• autodll-webpack-plugin - Share common deps with disk caching.

• Webpack@5 module disk cache (webpack/webpack#6527)

• cache-loader - Caches the result of following loaders on disk

• Prebuild next.js pages on server start

• Allow multiple pages in the same bundle

• Reduce size of common deps

  • Lodash = babel-plugin-lodash + lodash-webpack-plugin + https://medium.com/making-internets/why-using-chain-is-a-mistake-9bc1f80d51ba

Metrics

• Server start time.
• Incremental build time for changes on same page (HMR).
• Direct page visit (typing in url).
• Navigating to new page via a link.
• SSR of React Component

Profiling tools

• Chrome DevTools --inspect - profile any part of the app (including flamecharts with a slightly easier workflow than ProfilerPlugin I think)
• ProfilerPlugin in Webpack@4 - generates flamecharts for builds
• webpackDevMiddleware.stats = 'verbose'
• Monkey-patch next build to show stats.toString()
• WebpackAnalyzePlugin
• require('perf_hooks') - https://nodejs.org/api/perf_hooks.html

[timneutkens]

We're currently working on upgrading to webpack 5 incrementally. webpack 5 has a built-in disk cache which will help with a lot of the slowdowns that were previously noted with webpack 4.

[remorses]

I noticed that in developement some packages that expose both main and module entrypoints get loaded 2 times by webpack, the client webpack config loads the module esm entrypoint, the server webpack config loads the main cjs entrypoint instead

It would be faster to only use the main entrypoint in developement to not run loaders twice (use the same cache)

[timneutkens]

It would be faster to only use the main entrypoint in developement to not run loaders twice (use the same cache)

This is a misconception. The loaders have different settings between the 2 configs as well. node_modules are not compiled for the server bundle btw.

[malixsys]

This happens on first access, but can we trigger it via an api call to NextServer?

event - build page: /
event - compiled successfully

[timneutkens]

Now that Next.js 12 is out I'm going to close this issue: https://nextjs.org/blog/next-12#faster-builds-and-fast-refresh-with-rust-compiler

Note that closing this issue does not mean that we're going to stop optimizing Next.js performance, just that the things mentioned in this issue have been (mostly) implemented as well as many other optimizations that improve speed. We're still working on many other performance improvements on top of the new Rust-based compiler.

If you're experiencing slowdowns with your application please see #29559 (comment) where I'm keeping track of common cases that apps take longer to compile.

[balazsorban44]

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.