monorepo support

[chriseppstein]

In a monorepo I'd like a way to have a single source of truth for what versions are pinned. Can we have a way of extending notion config within a sub-project? E.g.

{
  "toolchain": {
    "extends": "../../package.json"
  }
}

This would still allow for overrides on a per-project basis but simplify the common case within a monorepo.

[dmfenton]

I think monorepos are already well supported. Works when moving between my 2 monorepos which have the node version pinned at the root.

cd ~/hub/indexer
➜  indexer git:(master) ✗ node --version
v10.16.0
➜  indexer git:(master) ✗ cd ~/hub/ui/packages/opendata-ui
➜  opendata-ui git:(f/dataset-search) ✗ node --version
v8.16.0

[charlespierce]

@dmfenton Which directory were you in when you ran those commands? If you were in the root of each monorepo it will probably work.

The issue that we have currently is that we detect the first package.json file in the directory structure as the package root. So in a monorepo with multiple package.json files, if the volta key is specified in only the root, we only work in folders where the root is the first parent with that file. But Volta won’t work correctly in the subrepositories without a bunch of duplication.

[dmfenton]

odd, it works when entering a package of one monorepo, but not the other

➜  ui git:(f/dataset-search) cd ~
➜  ~ node --version
v8.16.0
➜  ~ cd ~/hub/indexer/packages/monitor
➜  monitor git:(master) ✗ node --version
v8.16.0
➜  monitor git:(master) ✗ cd ~/hub/indexer
➜  indexer git:(master) ✗ node --version
v10.16.0
➜  indexer git:(master) ✗ cd ~/hub/ui/packages/content-library-engine
➜  content-library-engine git:(f/dataset-search) ✗ node --version
v8.16.0

[dmfenton]

I see why that was working now, the correct version was my default in the sub package where there is no Volta pinning.

➜  ember-arcgis-layout-cards git:(master) volta install node@12
success: installed and set [email protected] as default
➜  ember-arcgis-layout-cards git:(master) cd ~/hub/indexer
➜  indexer git:(master) ✗ node --version
v10.16.0
➜  indexer git:(master) ✗ cd ~/hub/ui/packages/layout-editor-engine
➜  layout-editor-engine git:(master) node --version
v12.4.0

Is it possible to ignore package.json files that do not contain Volta pinning and continue to traverse upwards? At least for some finite number of directories?

[charlespierce]

@dmfenton Yeah, that’s likely going to be a part of the solution for monorepo support.

We’ve got this tagged as “Needs RFC” because there are a lot of corner cases to consider, and we want to make sure that we tackle them correctly. Any input on the common use-cases for monorepos and how you would expect Volta to act would be helpful on that front.

[sclaxton]

Wanted to chime in on this issue that this gap in support also affects the Ember eco-system, particularly in-repo addons. In-repo addons are essentially "sub-projects" in the same way as yarn workspaces, et al. The only time you would really notice this is when you might want to run an in-repo addon's tests suite independently from the host app or build/serve up the addon's "dummy" app.

It seems to me, as a consumer, that volta should probably just have a config based escape hatch for pointing to the correct project root, as @chriseppstein originally suggested.

[runspired]

can we use git root to determine project root?

[chriseppstein]

@runspired it's not enough to just use the "root package.json". it's reasonable for some subprojects of a monorepo to have different volta configuration(s). For instance, css-blocks has a vscode plugin that I'd really like to pin to the current version of node that electron uses, while the rest of the project should be on an older version of node for compatibility reasons.

[wagenet]

My personal preference would just be to traverse upwards until it finds (or doesn't find) a specified version in a package.json.

[charlespierce]

@wagenet That's definitely a reasonable option. The only issue I see, and this would really take some discussion, is should we support "mixed" versions? That is, we currently let you specify both node and yarn in package.json, so should we let you specify node in one and yarn in a different file? If not, then I think your suggestion is very likely spot-on, but we don't have enough context at the moment to know if that behavior would be useful.

Perhaps we could start with that approach and then tackle any extra edge cases later so we can progressively add support for different monorepo use-cases.

[wagenet]

@charlespierce what about making it explicit as a first pass?

"volta": {
  "node": "inherit"
}

If it found "inherit" it would look up the tree. It would be nice not to have to add the volta declaration to all the workspace children, but this would have the benefit of not having to specify the same versions multiple times and possibly forgetting to update one.

[chriseppstein]

should we let you specify node in one and yarn in a different file?

Yes. Don't try to be smarter than the developer.

just ... traverse upwards

Please allow us to specify any json file containing {"volta": { ... } } it shouldn't have to be a package.json.

As I suggested above:

{
  "volta": {
    "extends": "../../package.json"
  }
}

is simple enough and it matches the syntax that is used in other projects such as eslint.

If you want to simplify the common case of walking up the directory tree, then a {"inherit": true} is fine, but I think it's more complexity than it's worth.

[wagenet]

I would also be fine with @chriseppstein's suggestion.

[rwjblue]

FYI - @charlespierce posted an RFC for this yesterday: volta-cli/rfcs#43. Please review and chime in!

[Offirmo]

Late to the party, I evaluated volta today and this is a pain.

nvm correctly looks upward until it finds .nvmrc, correctly inheriting unless explicitely asked not to.

With volta, I'll now have to add an unnecessary config: "volta": { "extends": "../../package.json" } to all my submodules package.json

IMO the extends should only be needed when one wants to locally override the versions.

[charlespierce]

Hi @Offirmo, thanks for the feedback. I definitely agree that needing to add extends to every submodule (including any new ones that area created in the future) is annoying and not great ergonomics. At the surface, I like the idea of only requiring extends if you want to override. My only concern is how we define the boundary of a monorepo to avoid pulling too much.

Part of that has to do with the fact that the data is stored in package.json, so it's not immediately clear when things should keep walking up the tree vs. when we should stop. One solution might be to introduce a separate file (See #282), which has the same semantics as .nvmrc, where it isn't directly tied to a single package (so we would search outside the boundaries of a "package" to find it).

[Offirmo]

Thanks for the answer!

Suggestion: how about supporting .nvmrc? This file is a quasi-standard, would be great for making volta a drop-in replacement! (can always be deprecated later)

[chriskrycho]

As @ljharb put it on #282:

I beg you not to support .nvmrc at all unless you can support 100% of its semantics, which include alias lookup, release line lookup, LTS lookup (lts/*, lts/argon, etc), major (4), minor (4.2), or exact (4.2.1).

That is, as you can imagine, a non-trivial undertaking!

Speaking for myself here (I have not conferred with other maintainers), I agree with the sentiment pretty strongly! In any case, #282 is the right place for further discussion of that approach specifically.