feat(npm): support Yarn 2 offline cache and zero-installs #7220
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,6 @@ | ||
| import path from 'path'; | ||
| import is from '@sindresorhus/is'; | ||
| import { parseSyml } from '@yarnpkg/parsers'; | ||
| import upath from 'upath'; | ||
| import { SYSTEM_INSUFFICIENT_DISK_SPACE } from '../../../constants/error-messages'; | ||
| import { id as npmId } from '../../../datasource/npm'; | ||
|
|
@@ -274,7 +275,7 @@ interface ArtifactError { | |
| stderr: string; | ||
| } | ||
|
|
||
| interface UpdatedArtifacts { | ||
| name: string; | ||
| contents: string | Buffer; | ||
| } | ||
|
|
@@ -336,9 +337,71 @@ async function resetNpmrcContent( | |
| } | ||
| } | ||
|
|
||
| // istanbul ignore next | ||
| async function updateYarnOffline( | ||
| lockFileDir: string, | ||
| localDir: string, | ||
| updatedArtifacts: UpdatedArtifacts[] | ||
| ): Promise<void> { | ||
| try { | ||
| const resolvedPaths: string[] = []; | ||
| const yarnrc = await getFile(upath.join(lockFileDir, '.yarnrc')); | ||
| const yarnrcYml = await getFile(upath.join(lockFileDir, '.yarnrc.yml')); | ||
|
|
||
| if (yarnrc) { | ||
| // Yarn 1 (offline mirror) | ||
| const mirrorLine = yarnrc | ||
| .split('\n') | ||
| .find((line) => line.startsWith('yarn-offline-mirror ')); | ||
| if (mirrorLine) { | ||
| const mirrorPath = mirrorLine | ||
| .split(' ')[1] | ||
| .replace(/"/g, '') | ||
| .replace(/\/?$/, '/'); | ||
| resolvedPaths.push(upath.join(lockFileDir, mirrorPath)); | ||
| } | ||
| } else if (yarnrcYml) { | ||
| // Yarn 2 (offline cache and zero-installs) | ||
| const config = parseSyml(yarnrcYml); | ||
|
There was a problem hiding this comment. Fwiw, the name is a bit misleading but it's regular YAML, so you can use any parser you want, not necessarily the Yarn one. Or you can use the There was a problem hiding this comment. @arcanis Thank you for your valuable comments!
renovate/lib/manager/npm/extract/yarn.ts Line 15 in c65d400 @yarnpkg/parsers seems to use js-yaml so I think it's OK and would give more consistent behavior with Yarn, e.g., empty file handling.
|
||
| resolvedPaths.push( | ||
| upath.join(lockFileDir, config.cacheFolder || './.yarn/cache') | ||
| ); | ||
|
|
||
| resolvedPaths.push(upath.join(lockFileDir, '.pnp')); | ||
| if (config.pnpDataPath) { | ||
| resolvedPaths.push(upath.join(lockFileDir, config.pnpDataPath)); | ||
| } | ||
| } | ||
| logger.debug({ resolvedPaths }, 'updateYarnOffline resolvedPaths'); | ||
|
|
||
| if (resolvedPaths.length) { | ||
| const status = await getRepoStatus(); | ||
| for (const f of status.modified.concat(status.not_added)) { | ||
| if (resolvedPaths.some((p) => f.startsWith(p))) { | ||
| const localModified = upath.join(localDir, f); | ||
| updatedArtifacts.push({ | ||
| name: f, | ||
| contents: await readFile(localModified), | ||
| }); | ||
| } | ||
| } | ||
| for (const f of status.deleted || []) { | ||
| if (resolvedPaths.some((p) => f.startsWith(p))) { | ||
| updatedArtifacts.push({ | ||
| name: '|delete|', | ||
| contents: f, | ||
| }); | ||
| } | ||
| } | ||
| } | ||
| } catch (err) { | ||
| logger.error({ err }, 'Error updating yarn offline packages'); | ||
| } | ||
| } | ||
|
|
||
| export interface WriteExistingFilesResult { | ||
| artifactErrors: ArtifactError[]; | ||
| updatedArtifacts: UpdatedArtifacts[]; | ||
| } | ||
| // istanbul ignore next | ||
| export async function getAdditionalFiles( | ||
|
|
@@ -347,7 +410,7 @@ export async function getAdditionalFiles( | |
| ): Promise<WriteExistingFilesResult> { | ||
| logger.trace({ config }, 'getAdditionalFiles'); | ||
| const artifactErrors: ArtifactError[] = []; | ||
| const updatedArtifacts: UpdatedArtifacts[] = []; | ||
| if (!packageFiles.npm?.length) { | ||
| return { artifactErrors, updatedArtifacts }; | ||
| } | ||
|
|
@@ -538,43 +601,7 @@ export async function getAdditionalFiles( | |
| name: lockFileName, | ||
| contents: res.lockFile, | ||
| }); | ||
| await updateYarnOffline(lockFileDir, config.localDir, updatedArtifacts); | ||
| } else { | ||
| logger.debug("yarn.lock hasn't changed"); | ||
| } | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd recommend to switch those two blocks (first check for
.yarnrc.yml, then for.yarnrc). In my work repo for instance, we have both a.yarnrcand a.yarnrc.yml, with the first one pointing to a binary explaining that the global Yarn isn't recent enough because it doesn't read.yarnrc.yml, and needs to be upgraded.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a minimum version of Yarn v1 that supports "bootstrapping" Yarn v2?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yarn 1 reading paths from
.yarnrc.ymlstarted with 1.18.0. Before that it was still possible, but you had to put the right path in both files.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the PR 😄