Packages from nodePackages_latest don't result in executables in $PATH.

[seanparsons]

Describe the bug

A shell including packages from nodePackages_latest like yarn or pnpm don't result in executables on the path the way nodePackages does.

Steps To Reproduce

Steps to reproduce the behavior:

1. Run nix-shell -p nodePackages_latest.yarn --run "yarn --version"

Expected behavior

I would expect to see the version number of yarn instead of /tmp/nix-shell-433770-0/rc: line 1: yarn: command not found.

Additional context

This works find if nix-shell -p nodePackages.yarn --run "yarn --version" is run instead.

I stumbled on this earlier but I'm not entirely certain if it's related: #142915

Notify maintainers

Apologies @marsam if you're not the most appropriate person to pull into this!

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.76, NixOS, 21.05.3990.372e59d2af7 (Okapi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.16`
 - channels(root): `"nixos-21.05.3990.372e59d2af7, nixos-hardware"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[seanparsons]

I've also attempted this with a more bleeding edge version of nixpkgs and it still appears to be an issue.

[andersk]

This seems to be a node2nix issue. Cc @svanderburg.

[happysalada]

node2nix only supports node_16 on master, the version included in nixpkgs does not support it.
Perhaps we could do a bump to unstable for node2nix ?

[andersk]

node2nix master still has this issue.

[miuirussia]

node2nix master still has this issue.

confirm it

[CMCDragonkai]

Yes node2nix master has this issue: svanderburg/node2nix#293.

However I'm not sure if this has anything to do with the OP's problem.

This is because nixpkgs master isn't even using 1.10.0 or 1.11.0: svanderburg/node2nix#291 (comment)

It claims it's on 1.10.0, but the actual executable is still 1.9.0 when you check the version.

[andersk]

@CMCDragonkai I suspect your test on 1.9.0 was tainted by the fact that 1.9.0 did not support --nodejs-16 (svanderburg/node2nix@e73c616), and you didn’t notice because 1.9.0 also did not complain about invalid options (svanderburg/node2nix#272). If you account for this (--pkg-name nodejs-16_x), you’ll see that 1.9.0 is equally affected, matching the OP’s problem.

[CMCDragonkai]

I'm not sure what OP's problem actually is because I'm not sure what the current nixpkg's master node packages are built with. As I said, the master claims that it has 1.10.0, but that package is actually 1.9.0. And there was a PR to update all of the node packages but that isn't even merged.

My experiments show that a combination of behaviour changes with npm install (from npm 6 to npm 8) and node2nix has resulted it in no longer making the bin executable symlinks. So I'm not sure what the fix will be.

While we wait on upstream to figure it out, I've ended up with duct-tape solution using jq to directly parse out the package.json information to produce executable symlinks: MatrixAI/TypeScript-Demo-Lib#37 (comment)

[andersk]

@CMCDragonkai I’m sure. master has 1.9.0 with some patches, see this override:

nixpkgs/pkgs/development/node-packages/default.nix

Lines 251 to 294 in 87d34a6

	node2nix = super.node2nix.override {
	buildInputs = [ pkgs.makeWrapper ];
	# We need to apply a patch to the source, but buildNodePackage doesn't allow patches.
	# So we pin the patched commit instead. The commit actually contains two other newer commits
	# since the last (1.9.0) release, but actually this is a good thing since one of them is a
	# Hydra-specific fix.
	src = applyPatches {
	src = fetchFromGitHub {
	owner = "svanderburg";
	repo = "node2nix";
	rev = "node2nix-1.9.0";
	sha256 = "0l4wp1131nhl9c14cn8bwawb8f77h1nfbnswgi5lp5m3kzkb27jn";
	};
	patches = [
	# remove node_ name prefix
	(fetchpatch {
	url = "https://github.com/svanderburg/node2nix/commit/b54d45207427ff46e90f16f2f32771fdc8bff5a4.patch";
	sha256 = "sha256-ubUdF0q3l4xxqZ7f9EiQEUQzyqxi9Q6zsRPETHlfzh8=";
	})
	# set meta platform
	(fetchpatch {
	url = "https://github.com/svanderburg/node2nix/commit/58736093161f2d237c17e75a96529b018cd0ac64.patch";
	sha256 = "0sif7803c9g6gjmmdniw5qxrq5igiz9nqdmdrcf1hxfi5x43a32h";
	})
	# Extract common logic from composePackage to a shell function
	(fetchpatch {
	url = "https://github.com/svanderburg/node2nix/commit/e4c951971df6c9f9584c7252971c13b55c369916.patch";
	sha256 = "0w8fcyr12g2340rn06isv40jkmz2khmak81c95zpkjgipzx7hp7w";
	})
	# handle package alias in dependencies
	# https://github.com/svanderburg/node2nix/pull/240
	#
	# TODO: remove after node2nix 1.10.0
	(fetchpatch {
	url = "https://github.com/svanderburg/node2nix/commit/644e90c0304038a446ed53efc97e9eb1e2831e71.patch";
	sha256 = "sha256-sQgVf80H1ouUjzHq+2d9RO4a+o++kh+l+FOTNXfPBH0=";
	})
	];
	};
	postInstall = ''
	wrapProgram "$out/bin/node2nix" --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.nix ]}
	'';
	};

It is easy to verify that 1.9.0 is equally affected without relying on nixpkgs:

$ echo `["yarn"]` > node-packages.json

$ npx [email protected] --version
Need to install the following packages:
  [email protected]
Ok to proceed? (y) y
node2nix 1.9.0

$ npx [email protected] --pkg-name nodejs-16_x --input node-packages.json 
info attempt registry request try #1 at 12:10:57 PM
http request GET https://registry.npmjs.org/yarn
http 200 https://registry.npmjs.org/yarn

$ nix-build -A yarn
…

$ ls result/
lib

$ npx [email protected] --pkg-name nodejs-14_x --input node-packages.json 
info attempt registry request try #1 at 12:14:27 PM
http request GET https://registry.npmjs.org/yarn
http 200 https://registry.npmjs.org/yarn

$ nix-build -A yarn
…

$ ls result/
bin  lib

Same results with 1.9.0, 1.10.0, and 1.11.0. It really is a difference between --pkg-name nodejs-14_x and --pkg-name nodejs-16_x, not a difference between node2nix versions.

In nixpkgs, it’s the difference between nodePackages and nodePackages_latest for the same reason, because nodePackages is defined with .override { nodejs = nodejs-14_x; }, while nodePackages_latest gets nodejs_latest which is nodejs-16_x.

[lilyinstarlight]

This should be fixed by #193337, if anyone would like to test (nodePackages.yarn at least seems to work with nodejs-18_x on my system now)

[utybo]

I can confirm this is fixed, I no longer need to create symlinks manually for node packages, thank you very much to the devs who fixed this!