feat: experimental server persistent caching #10671

Akryum · 2022-10-27T22:14:40Z

Description

I have seen loading speed increase of up to 5x faster in subsequent warm server starts on my computer. It's basically almost as fast as a page reload.

Try it

You can install it with a special release at @akryum/vite@server-cache:

npm i -D vite@npm:@akryum/vite@server-cache
yarn add -D vite@npm:@akryum/vite@server-cache
pnpm i -D vite@npm:@akryum/vite@server-cache

It can be enabled with the new experimental.serverPersistentCache config.

import { defineConfig } from 'vite'

export default defineConfig({
  experimental: {
    serverPersistentCache: true
  }
})

You can further customize the feature with an object:

import { defineConfig } from 'vite'

export default defineConfig({
  experimental: {
    serverPersistentCache: {
      enabled: true,
      cacheVersion: 'MeowV1',
      cacheVersionFromFiles: [
        'some-config-file.js',
      ],
      exclude: (url) => url.includes('@imagetools') || url.includes('node_modules') || !!url.match(/\.(jpg|webp)/),
      cacheDir: 'cache-vite-server-1',
    }
  }
})

If you encounter issues try clearing the cache:

rm -rf ./node_modules/.vite/server-cache

Known issues

~~Page load might fail if the optimizeDeps cache (node_modules/.vite) is empty (workaround is to clear persistent cache and try again)~~
~~Cached module HMR~~ Now working!

Demo

Screencast.from.28-10-2022.00.09.52.webm

On the app I'm testing this against, the page load goes from ~10s (cold start) to ~2s (cache hit).

Additional context

This is a PoC about using persistent caching in development.

The idea is to cache as close as possible to the slowest part of the module processing: await pluginContainer.transform(), in order to avoid breaking stuff as much as possible.

Implementation notes:

Cache writes code and source map content directly in separate files to avoid JSON parsing as much as possible
Cache writes and reads are lazy on a per-module basis.
There is a manifest storing useful data about the persistent cache.
The cache has an automatic busting logic based on the config cacheVersion and cacheVersionFromFiles (using content hashing). By default it also adds the vite version, the vite config file, the define values, the env values, the package lock file and the tsconfig.json file.
There is a special logic to automatically cache await pluginContainer.load() logic in case side effects from await pluginContainer.transform() would affect it. For example, the Svelte plugin adds CSS subrequests in an internal cache as a side-effect of transform().
HMR on cached modules works by restoring a few properties to the Module Graph Node such as importedModules.
Imports of optimized deps are rewritten using the final browserHash so that they are correct for the next server start.
I have added a few logs that could help debugging issue while the feature is experimental. Most could then be removed.

What is the purpose of this pull request?

Bug fix
New Feature
Documentation update
Other

Before submitting the PR, please make sure you do the following

Read the Contributing Guidelines.
Read the Pull Request Guidelines and follow the Commit Convention.
Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
Ideally, include relevant tests that fail without this PR but pass with it.

Akryum · 2022-10-28T00:02:51Z

Will probably not work with Svelte as they use a custom cache for the .svelte file CSS requests:
https://github.com/sveltejs/vite-plugin-svelte/blob/49f80e75264343032fccb5e8939168174e65bc91/packages/vite-plugin-svelte/src/index.ts#L111

Akryum · 2022-10-28T00:44:46Z

Found a solution: cache the loadResult and restore it in case pluginContainer.load() didn't yield a result.

In the Svelte plugin:

This is what happens when loading the .svelte file CSS: load('SystemDialog.svelte?svelte&type=style&lang.css') => internalCache.getCSS(id)

Solution:

Cache miss
- Persist loadResult and save the relation with the SystemDialog.svelte file into the cache manifest
Cache hit
- internalCache.getCSS('SystemDialog.svelte?svelte&type=style&lang.css') will return null since transform('SystemDialog.svelte') was never called (it's where the .svelte file is split in multiple parts into the plugin internalCache)
- Restore the previously cached loadResult to get the correct code (only the <style> part)

I tested on this svelte app:
https://github.com/PuruVJ/macos-web/

Cold start (no cache):

Warm start (cache hit):

Akryum · 2022-10-28T16:50:09Z

Tested with https://github.com/martpie/museeks (electron + React)

Cold start:

Warm start:

Akryum · 2022-10-28T17:24:59Z

Tested with https://github.com/yyx990803/vite-vs-next-turbo-hmr (React)

Cold start:

Warm start:

Akryum · 2022-10-28T17:37:27Z

Test on https://github.com/noeldemartin/umai (Vue)

Cold start:

Warm start:

Akryum · 2022-10-29T20:36:56Z

Tested with yyx990803/vite-vs-next-turbo-hmr (React)

So I have tested again but on Chrome as I had a file descriptor limit issue.

Cold start:

Warm start:

ArnaudBarre

Thanks a lot for this work. The api with custom exclude and config files is really what was missing in my implementation. And the lazy loading of cache values is a lot more coherent with the vite architecture.

~~I think what is missing is a check for environment variables. A simple first solution could be to exclude from the cache files that contains pattern like process.env or import.meta.env~~

packages/vite/src/node/server/persistentCache.ts

ArnaudBarre · 2022-10-31T21:01:17Z

packages/vite/src/node/server/transformRequest.ts

+    } else {
+      const saveKey = fileCacheInfo.relatedModules[loadCacheKey]
+      if (saveKey) {
+        loadResult = await _persistentCache.read(saveKey)


Can you explain a bit more the goal of this whole block? (maybe with an example of two ids for svelte)

I don't understand why the override of await pluginContainer.load(id, { ssr }) is only done when idWithoutHmrFlag !== file

Maybe a first version could only focus on the transform part and see if some plugins can migrate the heavy work from load to transform?

The goal is to restore the result of plugin.load in case it needed side effects from plugin.transform. For example (not real requests):

transform: Foo.svelte => put <style> part into an internal cache as side-effect

load: Foo.svelte?type=css => retrieve the content of <style> from the internal cache.

I will bootstrap the a simple svelte template with the inspect plugin to better understand how it works and comeback here after.

As I understand it, the load hook will return nothing for Foo.svelte?type=css when Foo.svelte was already cached (and thus, never processed by transform pipeline). By caching the load result, Vite can still access the Foo.svelte?type=css contents for the cached Foo.svelte module.

Since the load hook is always called (even for cached modules), this is a necessary workaround.

Ideally, there would be a way for a load result to declare a "dependency" on another file's contents. This way, Vite wouldn't have to cache the load result of every file.

edit: Oh I just remembered that we have addWatchFile from the Rollup API. That could probably work. 🤔

packages/vite/src/node/config.ts

packages/vite/src/node/server/persistentCache.ts

packages/vite/src/node/server/transformRequest.ts

benmccann · 2022-11-05T13:53:13Z

I wonder about the option name actually. It's sort of an implementation detail that the cache is on the server object. Perhaps just persistentCache or devPersistentCache? The former would allow a dev vs build option in case we ever wanted to add build caching in the future

benmccann · 2022-11-05T13:56:50Z

packages/vite/src/node/server/transformRequest.ts

+  const includedInPersistentCache =
+    _persistentCache &&
+    !file.includes(server.config.cacheDir) &&
+    !file.includes('vite/dist/client') &&


this line could use a comment as to either when it would occur or why we want to exclude it

Well it excludes the vite client

Well it excludes the vite client

That doesn't explain why 😆

packages/vite/src/node/server/transformRequest.ts

Akryum · 2022-11-07T09:44:00Z

I wonder about the option name actually. It's sort of an implementation detail that the cache is on the server object. Perhaps just persistentCache or devPersistentCache? The former would allow a dev vs build option in case we ever wanted to add build caching in the future

Yes it's server as opposed to build. I guess it could be under server if moved out of experimental.

bluwy · 2022-11-18T10:24:58Z

In the last meeting while discussing #10943, we thought it would be nice if all the caching logic is handled as an internal Vite plugin. It's not completely necessary for this PR but it would make it easier to maintain. It might involve adding hooks/options/metadata for the Vite plugin lifecycle, but they should be kept internal only.

arilotter · 2022-12-12T17:07:05Z

Hey @Akryum - we've been testing your branch on our codebase, and have run into an issue; refreshes after code has changed break the dev server if your code has circular imports.
I wish my codebase didn't have circular imports, but we have thousands, so fixing them quickly sn't very viable 😓

Here's a minimal reproduction! https://github.com/arilotter/vite-caching-circular-imports

Let me know if I can help 😁

patak-dev · 2022-12-14T13:34:18Z

Leaving a comment for reference, this PR has proven that server persistent caching could bring a big perf boost to some projects. It wasn't merged as part of Vite 4 because we were unsure of the current caching invalidation logic being correct for all cases.

I think we have two options moving forward (I think we should pursue both):

Re-evaluate the idea from @aleclarson of providing a caching hooks API so this PR or other experimentation can be implemented by plugins (with caching logic that may work only for certain kinds of projects, that is totally fine for a plugin)
Keep working on identifying perf bottlenecks and resolving them instead of relying on caching. Experimentation from @ArnaudBarre points in this direction as a way to improve cold start times. Some big improvements could come from:
- Replace magic string with a Rust alternative (and in general help Lukas to port parts of rollup to native).
- Replace postCSS with Lighting CSS.
- Implement the SSR transformation on native.

rtsao · 2022-12-15T19:35:18Z

IMHO persistent caching in core might be preferable, even if it is an opt-in experiment and disabled by default initially.

A core value proposition of Vite is fast performance and currently SSR startup is a major bottleneck at the moment
It may be more difficult to implement caching as a plugin instead and ultimately expanding the plugin API surface area has costs as well.

Support for a basic level of caching that works for every project seems far more valuable than supporting niche caching logic for specific applications and circumstances.

If there's still desire to have caching in core, I can work on adapting the bug reported by @arilotter as a failing test and try fixing it.

luxaritas · 2022-12-15T19:42:52Z

Not knowing the details of the situation here, I also want to ask if being "unsure of the current caching invalidation logic being correct for all cases" is the issue, why isn't one of the potential avenues to pursue making it sufficiently robust? Or has there been some situation identified where it'd be impossible for Vite to cover all its bases (presumably even with configuration)?

patak-dev · 2023-04-06T20:24:10Z

@Akryum would it be possible to re-test some of the examples you tried here using [email protected]? It would be great to know if the latest performance improvements reduce the need for caching, or how far we are from the ideal of a cached warm start to keep digging (or re-evaluate caching once more if we can't find a way to further speed up the dev server).

luxaritas · 2023-04-06T21:31:25Z

or re-evaluate caching once more if we can't find a way to further speed up the dev server

Apologies if I'm misreading, but if it isn't being considered, please also keep in mind the potential impact of caching on full builds, not just the dev server. This has the potential to be important eg for monorepo tools that rely on full rebuilds of dependency packages on change (eg, nx watch in a package-based monorepo)

caghand · 2023-04-27T00:54:51Z

@Akryum would it be possible to re-test some of the examples you tried here using [email protected]? It would be great to know if the latest performance improvements reduce the need for caching, or how far we are from the ideal of a cached warm start to keep digging (or re-evaluate caching once more if we can't find a way to further speed up the dev server).

@patak-dev, as someone who is fetching ~6000 modules per page, I can say that the speed increase in Vite 4.3.x is absolutely incredible. It's around 5 times faster! I could not believe it at first, but it's all real. It will make a huge difference to our productivity.
Personally, I had been eagerly waiting for this "persistent caching" functionality, but now I don't need it. :)

patak-dev · 2023-05-03T08:39:18Z

@caghand thanks for getting back with metrics for your app! Happy to see 4.3 has such a big effect in your case. Would it be possible to share CPU profiles of cold and warm start with us? We should continue to check if there are other perf gains out there. You can use pnpm run dev --force --open --profile then p, and the same without --force for warm start. A CPU profile will be generated at root. You could also use https://github.com/antfu/vite-plugin-inspect which will show you times for each plugin you use. You may be able to identify bottlenecks in some community plugins (and we can work together with them to improve them)

Akryum · 2023-09-08T23:53:58Z

Continued in #14333

Akryum added 5 commits October 28, 2022 00:12

feat: experimental server persistent caching

14bde2b

fix: check if config is nullish

c309eed

fix: don't cache modules that contains 'import.meta.glob'

57f081d

feat: exclude

ce26960

fix: add id in cache key

12216c3

Akryum added 2 commits October 28, 2022 02:10

refactor: includedInPersistentCache

a8f8d28

fix: cache loadResult to support svelte

fc73790

Akryum added 4 commits October 28, 2022 03:17

fix: resolve versionFromFiles path from resolvedRoot

061e645

fix: not waiting for clearing the cache files

d128365

feat: clearing cache log

138afc0

feat: clearer generated cacheVersion

9767372

Akryum added 4 commits October 29, 2022 01:07

feat: auto include tsconfig in versionFromFiles

228670c

refactor: move resolve options from config to persistentCache

d45302c

feat: save request id in manifest

f4ff16b

fix: manifest write debounce

8c512e6

Akryum added 4 commits October 29, 2022 23:36

feat: cleanup stale cache files

5750ddb

feat: cached modules HMR support

8c86c75

fix: disable cache file cleanup for now

830b5fb

fix: state cache files cleanup

0410cd3

ArnaudBarre reviewed Oct 31, 2022

View reviewed changes

Akryum added 4 commits November 1, 2022 04:10

feat: store url too

f4b2e5e

refactor: simplify resolvePersistentCacheOptions

8dd8dd7

refactor: remove cleanup

967d8cd

refactor: debug log

6b32599

Akryum added 6 commits November 4, 2022 11:23

chore: more debug logs

c1d9f8b

fix: temporary cache for patched modules

99a35ad

refactor: use a skip list instead of temp cache

037972f

chore: typo

466e479

fix: ssr code cache write

f0da21f

chore: add more debug log

31eaebe

benmccann reviewed Nov 5, 2022

View reviewed changes

Akryum added 3 commits November 7, 2022 10:57

refactor: getKey add ssr param

84d7a70

refactor: rename config to serverPersistentCache

ed4d3b2

chore: config.cacheDir comment

373d02d

This comment was marked as outdated.

Sign in to view

aleclarson mentioned this pull request Nov 15, 2022

Add preTransform and postTransform hooks #10943

Closed

4 tasks

patak-dev added the on hold label Dec 14, 2022

meedfine mentioned this pull request Jan 31, 2023

Dev open slow：Can you quickly compute the hash while parsing the request to return 304 #11859

Closed

4 tasks

benmccann added the performance Performance related enhancement label Mar 14, 2023

benmccann mentioned this pull request Apr 26, 2023

Cache generated images? JonasKruckenberg/imagetools#522

Closed

Akryum mentioned this pull request Sep 8, 2023

feat: persistent cache plugin #14333

Draft

9 tasks

Akryum closed this Sep 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: experimental server persistent caching #10671

feat: experimental server persistent caching #10671

Akryum commented Oct 27, 2022 •

edited

Loading

Akryum commented Oct 28, 2022 •

edited

Loading

Akryum commented Oct 28, 2022 •

edited

Loading

Akryum commented Oct 28, 2022

Akryum commented Oct 28, 2022

Akryum commented Oct 28, 2022

Akryum commented Oct 29, 2022

ArnaudBarre left a comment •

edited

Loading

ArnaudBarre Oct 31, 2022 •

edited

Loading

Akryum Nov 1, 2022

ArnaudBarre Nov 1, 2022

aleclarson Nov 15, 2022

aleclarson Nov 15, 2022 •

edited

Loading

benmccann commented Nov 5, 2022 •

edited

Loading

benmccann Nov 5, 2022 •

edited

Loading

Akryum Nov 7, 2022

aleclarson Nov 16, 2022

Akryum commented Nov 7, 2022 •

edited

Loading

This comment was marked as outdated.

bluwy commented Nov 18, 2022

arilotter commented Dec 12, 2022

patak-dev commented Dec 14, 2022

rtsao commented Dec 15, 2022 •

edited

Loading

luxaritas commented Dec 15, 2022

patak-dev commented Apr 6, 2023

luxaritas commented Apr 6, 2023

caghand commented Apr 27, 2023 •

edited

Loading

patak-dev commented May 3, 2023

Akryum commented Sep 8, 2023

feat: experimental server persistent caching #10671

feat: experimental server persistent caching #10671

Conversation

Akryum commented Oct 27, 2022 • edited Loading

Description

Try it

Known issues

Demo

Additional context

What is the purpose of this pull request?

Before submitting the PR, please make sure you do the following

Akryum commented Oct 28, 2022 • edited Loading

Akryum commented Oct 28, 2022 • edited Loading

Akryum commented Oct 28, 2022

Akryum commented Oct 28, 2022

Akryum commented Oct 28, 2022

Akryum commented Oct 29, 2022

ArnaudBarre left a comment • edited Loading

Choose a reason for hiding this comment

ArnaudBarre Oct 31, 2022 • edited Loading

Choose a reason for hiding this comment

Akryum Nov 1, 2022

Choose a reason for hiding this comment

ArnaudBarre Nov 1, 2022

Choose a reason for hiding this comment

aleclarson Nov 15, 2022

Choose a reason for hiding this comment

aleclarson Nov 15, 2022 • edited Loading

Choose a reason for hiding this comment

benmccann commented Nov 5, 2022 • edited Loading

benmccann Nov 5, 2022 • edited Loading

Choose a reason for hiding this comment

Akryum Nov 7, 2022

Choose a reason for hiding this comment

aleclarson Nov 16, 2022

Choose a reason for hiding this comment

Akryum commented Nov 7, 2022 • edited Loading

This comment was marked as outdated.

bluwy commented Nov 18, 2022

arilotter commented Dec 12, 2022

patak-dev commented Dec 14, 2022

rtsao commented Dec 15, 2022 • edited Loading

luxaritas commented Dec 15, 2022

patak-dev commented Apr 6, 2023

luxaritas commented Apr 6, 2023

caghand commented Apr 27, 2023 • edited Loading

patak-dev commented May 3, 2023

Akryum commented Sep 8, 2023

Akryum commented Oct 27, 2022 •

edited

Loading

Akryum commented Oct 28, 2022 •

edited

Loading

Akryum commented Oct 28, 2022 •

edited

Loading

ArnaudBarre left a comment •

edited

Loading

ArnaudBarre Oct 31, 2022 •

edited

Loading

aleclarson Nov 15, 2022 •

edited

Loading

benmccann commented Nov 5, 2022 •

edited

Loading

benmccann Nov 5, 2022 •

edited

Loading

Akryum commented Nov 7, 2022 •

edited

Loading

rtsao commented Dec 15, 2022 •

edited

Loading

caghand commented Apr 27, 2023 •

edited

Loading