Version upgrades and yarn.lock

[eliperelman]

There seems to be something out of step with bumping versions with dependent sub-packages. Let's take the following example:

package alpha:

{
  "name": "alpha",
  "version": "1.0.0"
}

package beta:

{
  "name": "beta",
  "version": "1.0.0",
  "dependencies": {
    "alpha": "1.0.0"
  }
}

If I want to make updates to alpha, I have to manually update the dependency version in beta to pick up the changes. If alpha has new dependencies, and thereby a new yarn.lock file, those deps won't be picked up in the yarn.lock file of beta.

Running bootstrap after this work doesn't make any changes either.

[guigrpa]

In your example, I guess beta depends on alpha, right?

Could you please provide a more specific example isolating your case as far as possible? It could be a bug, or an inherent limitation on how yarn link works.

[guigrpa]

Note that oao basically runs yarn link alpha from beta, and that this currently symlinks the entire alpha directory inside beta's node_modules. This operation changes nothing in beta's yarn.lock.

There's currently some discussion (yarnpkg/yarn#1213 and yarnpkg/rfcs#41) on how yarn link should work, and even a proposal for an improved yarn knit that may be what we want. I propose to track those developments and modify oao eventually, if needed.

[eliperelman]

Yes, beta would depend on alpha. I understand that during development we are using yarn link. Maybe an interim solution is to publish using a dependency tree just like bootstrap. For example:

project/
└ packages/
   ├ alpha/
   ├ beta/
   ├ gamma/
   └ delta/

Dependencies go gamma -> beta -> alpha. delta depends on no one. If instead of just publishing linearly, you publish in the order of:

1. Publish delta
2. Publish alpha
3. yarn upgrade alpha in beta. Commit updated package.json and yarn.lock. Publish beta.
4. yarn upgrade beta in gamma. Commit updated package.json and yarn.lock. Publish gamma.
5. Probably re-link after all publishes are done.

That is how I see it working theoretically. Let me know what you think of that idea.

[guigrpa]

I really want to keep oao simple and predictable:

• Bootstrap: yarn install external deps + yarn link internal ones (it doesn't build a dependency tree; it runs in two stages to keep linking simple).

• Publish: update sub-package versions when they've changed + git commit + git push + npm publish.

The proposed new publishing algorithm would involve multiple additional steps, knowledge of the transitive dependency graph, automatic internal dependency upgrades (which might not be what the user wants), more complex error handling and potentially unpredictable states. All this complexity might make it more error-prone.

Instead, I'd rather wait and see how yarn link (or yarn knit 😄) evolves (yarnpkg/yarn#1213 and yarnpkg/rfcs#41), and even whether lerna is finally absorbed by yarn, as this thread suggests.

In the meantime, consider the fact that yarn.lock files in libraries (not applications) are useful for development (so that everyone has the same environment), but are not taken into account for the final version resolution in end-user applications. Some conclusions from this:

• A user of your monorepo's sub-packages will be unaffected by what you specify in your yarn.lock files.

• A developer working on your monorepo, or a user of an application sub-package that may happen to be inside the monorepo (not the case in my storyboard monorepo nor (I guess) in your neutrino-dev) will oao bootstrap it and get a predictable set of dependencies all the way down: external ones are set in stone in the corresponding yarn.lock files, and internal ones are frozen for a particular monorepo version (git tag).

I'd agree we are missing a small piece of the puzzle: a --production flag for oao bootstrap (for the case in which there are application sub-packages in the monorepo). I open an issue for that.

P.S. Sorry about this long answer. What do you think about all this?

[eliperelman]

I think your answer is reasonable, although I approached it the way I did because I faced actual errors during development. With the lock files out of step locally, I couldn't run yarn outdated because yarn complained that the lock file was out of date. My guess is it was an issue with the link interfering with the integrity of node_modules during the outdated check. Anyway, I'll think about your proposal, and see what I can do locally to ease the publishing flow. Thanks!

[guigrpa]

Both --production (or NODE_ENV=production) and oao outdated are available in v0.5.7 😉.

[guigrpa]

Closing this, considering that --production and oao outdated seem to be useful for this.