Implements package extensions

[arcanis]

What's the problem this PR addresses?

Some packages have improper declarations, requiring to either send a PR upstream (and waiting for it to be accepted), or manually patching the lockfile (which feels a bit dirty). We should have a better story to encourage people to unblock themselves.

How did you fix it?

Yarn now supports a packageExtensions object in the .yarnrc.yml file which makes it possible to extend package declarations with new metadata. For example, the following rc would allow webpack to access lodash:

packageExtensions:
  "webpack@*":
    dependencies:
      lodash: ^4.17.15

The key is expected to by a semver range which will be matched against the effective version of the package (regardless of how the dependency got fetched - git dependencies will be semver-matched all the like).

This is a quite complicated fix because it involves making subtle changes to how the lockfile is generated. Before, Yarn used to store the resolved tree as-is - so just loading the tree (and running the virtual hydratation, since virtual packages are computed separately) used to be enough to get the representation of the dependency tree. This way nice, but it had a major drawback: making changes to the way the dependencies were resolved (for example adding a dependency, or using the resolutions field to remap them) caused the changes to be persisted inside the lockfile. Even after the edit sources were removed from the configuration the overrides were kept. It could be quite confusing, and made such features destructive by nature.

So instead, I reworked the lockfile to instead store the original version of the package definitions, before any normalisation or patching. This implies a few things:

• The lockfile doesn't reference anymore the npm: protocol in the dependencies field, because this field doesn't get normalised anymore.

  • Resolved ranges (ie the keys of the metadata in the lockfile) do reference the npm: protocol, because they are the fully normalised .

• Directly accessing storedPackages after instantiating a Project won't work, because at this point the Project will only contain the original versions of the package - before they have been normalised. In order to make it available, the consumer will have to call resolveEverything (ideally with the lockfileOnly flag) - cf yarn bin for an example.