[Bug?]: Yarn fails to handle when cacheFolder is a symlink farm #3514
Labels
Comments
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
This issue reproduces on master: |
adrian-gierakowski
added a commit
to rhinofi/yarn-plugin-nixify
that referenced
this issue
Jan 28, 2022
When used with pnp, we can avoid copying deps to the main project, so that rebuilds are faster and less spaces is used when only project source changes (but not deps). To make this work, we need to patch .pnp.cjs, since unfortunately symlinks don't work as packageLocations cannot be (see: yarnpkg/berry#3514). We also cannot use absolute paths, so we end up using relative paths to store locations containing each deps. This also works for unplugged deps.
adrian-gierakowski
added a commit
to rhinofi/yarn-plugin-nixify
that referenced
this issue
Jan 28, 2022
When used with pnp, we can avoid copying deps to the main project, so that rebuilds are faster and less spaces is used when only project source changes (but not deps). To make this work, we need to patch .pnp.cjs, since unfortunately symlinks don't work as packageLocations cannot be (see: yarnpkg/berry#3514). We also cannot use absolute paths, so we end up using relative paths to store locations containing each deps. This also works for unplugged deps.
2 tasks
|
It seems that this used to be supported but the functionality was removed in 22ef17a for performance reasons. I'm not quite sure removing the symlink resolution actually makes things any faster in the common case (when it's not a symlink) so maybe we can bring it back? |
3 tasks
github-merge-queue bot
pushed a commit
that referenced
this issue
Nov 18, 2024
This is the same as #5474 but allows edits by maintainer **What's the problem this PR addresses?** <!-- Describe the rationale of your PR. --> Yarn shouldn't care if zips in the cache are symlinks or not. The only reason it doesn't work is that finding the mount point skips zips if they're not regular files. This used to be supported until it was removed for "performance" reasons in #1474. The bulk of the logic in the linked PR involves resolving the real path of the symlink but I don't think that's needed for this? <!-- Link all issues that it closes. (Closes/Resolves #xxxx.) --> Resolves #3514 Closes #5474 (supersedes it) **How did you fix it?** I switched the check to use `stat` from `lstat`. **Checklist** <!--- Don't worry if you miss something, chores are automatically tested. --> <!--- This checklist exists to help you remember doing the chores when you submit a PR. --> <!--- Put an `x` in all the boxes that apply. --> - [X] I have read the [Contributing Guide](https://yarnpkg.com/advanced/contributing). <!-- See https://yarnpkg.com/advanced/contributing#preparing-your-pr-to-be-released for more details. --> <!-- Check with `yarn version check` and fix with `yarn version check -i` --> - [X] I have set the packages that need to be released for my changes to be effective. <!-- The "Testing chores" workflow validates that your PR follows our guidelines. --> <!-- If it doesn't pass, click on it to see details as to what your PR might be missing. --> - [x] I will check that all automated PR checks pass before the PR gets reviewed. --------- Co-authored-by: merceyz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Self-service
Describe the bug
When
cacheFolderpoints to a folder containing symlinks to zips rather than actual zips, Yarn fails to handle it as we expect. The failure happens regardless of whether--immutable-cacheis used. Our main use case is withnodeLinker: node-modules, but a similar error also appears to affect PnP. When thecacheFolderis a symlink farm,yarnfails with the following error:Here is the additional failure case affecting PnP. This doesn't affect us right now, but our goal is to migrate to PnP in time, and this would impact our ability to move:
Reported error:
To reproduce
Environment
System: OS: macOS 11.5.2 CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz Binaries: Node: 14.16.1 - /private/var/folders/4y/p9httzjd7px11pkmsfvlbdym0000gp/T/xfs-1a58a4dc/node Yarn: 3.1.0-rc.8.dev - /private/var/folders/4y/p9httzjd7px11pkmsfvlbdym0000gp/T/xfs-1a58a4dc/yarn npm: 6.14.12 - ~/.nvm/version/npmAdditional context
We use Bazel (though not rules_nodejs). For each build action, Bazel naturally provides a sandboxed symlink farm because it doesn't guarantee that the folder the real zips are in is sandboxed. We'd like to point Yarn to this symlink farm instead of the real folder and we'd also like to avoid having to dereference the links and pay the cost of writing it to disk. For us, this only impacts install for now since we're using
nodeLinker: node-modules, and after the install, thecacheFolderhas no impact. But this will become increasingly important for us in order to migrate to PnP, otherwise we have to dereference the symlink farm at the beginning of every action.The text was updated successfully, but these errors were encountered: