Separate lockfile for each module in a monorepro but with a shared virtual store

[chankev]

We currently have a large monorepro with 150+ modules that shares one lock file. As a result, we often see lock file merge conflicts and updates to one module's package.json affecting another module because the shared lock file is updated.

To prevent this, we would like to create separate lockfiles, but with a shared virtual store. The shared virtual store is required because, in our testing, webpack correctly consolidates duplicate dependencies in a shared virtual store. When the virtual store was separate, the resulting JS bundle had duplicate dependencies.

We tried two methods:

Method 1: We set shared-workspace-lockfile in .npmrc to false, but that seems to result in a separate virtual store for each module.

Method 2: We tried running 'pnpm install --virtual-store-dir='/same/directory/.pnpm'' on each module folder, renaming the resulting virtual .pnpm folder so it doesn't get cleared when 'pnpm install' is ran on the next module folder, and then merging all the renamed .pnpm folders together in the end. This appears to allow separate lockfiles while also keeping the same virtual store.

However, before we proceed further, we were curious if anyone was aware of issues/concerns from sharing the same virtual store but different lockfiles, or if there is a native way to do this? Alternatively, if webpack has a way to consolidate dependencies in separate virtual stores, we would be open to exploring that as well.

Thanks!

[gluxon]

This is an interesting idea to reduce merge conflicts.

One issue I do foresee is that pnpm "cleans up" the virtual store directory at a configured interval. When this happens, running pnpm install for one pnpm-lock.yaml may delete the modules from the virtual store that another part of your repo is still relying on. You could consider setting modules-cache-max-age to a higher number than the default of 7 days, but this means your .pnpm dir will continue growing and contain unnecessary dependencies for a longer period of time.

I'm a pnpm user myself in a monorepo with 1000+ packages. I'd be interested in hearing what kinds of merge conflicts you see in pnpm-lock.yaml the most. I've been investing in reducing pnpm-lock.yaml merge conflicts along with help from other contributors as well. (Example #5091)