[BUG] Arguments are not correctly passed from CLI to npm script (npm 7, Windows, Powershell)

[cyberbiont]

Environment:

• OS: Windows 10Pro 20H4
• Node: 16.0.0
• npm: 7.11.0

package.json:

...
"script": { 
    "git": "git",
    "help": "npm run git -- --help"
}

npm run help // I get help about Git (as intended)
npm run git -- --help from Powershell // I get help about npm run-script

As far as I can tell, the problem exists only in Powershell on Windows.
cmd and Git Bash work as intended.
Bash and pwsh on Linux also work as intended.

The problem is new to npm7 (I switched back to npm6 and it worked as intended).

[rikbrowning]

I have noticed the following:

running npm run test:unit -- TestName will pass the argument to the underlying command
running npm run test:unit -- -u TestName will not pass the -u but still passes TestName.

[holtalanm]

I am experiencing this same issue.

npm version: 7.19.0
node versiion: 16.4.0
powershell version: 5.1.19041.610
os: windows v10.0.19042.0

when running npm run test:unit -- -h, I get:

npm run-script

Run arbitrary package scripts

Usage:
npm run-script <command> [-- <args>]

Options:
[-w|--workspace <workspace-name> [-w|--workspace <workspace-name> ...]]
[-ws|--workspaces] [--if-present] [--ignore-scripts]
[--script-shell <script-shell>]

aliases: run, rum, urn

Run "npm help run-script" for more info

when running npm run test:unit -- -- -h, I get:

> vue-cli-service test:unit "-h"


  Usage: vue-cli-service test:unit [options] <regexForTestFiles>

  Options:

    --watch   run tests in watch mode

  All jest command line options are supported.
  See https://facebook.github.io/jest/docs/en/cli.html for more details.

It seems a workaround currently is to include an additional -- in the arguments to npm run when running on powershell.

can also confirm that this behavior is new to npm7, as I just upgraded from v6 recently.

[holtalanm]

after completely uninstalling nodejs and all globally installed packages, then reinstalling the latest version of nodejs, this issue went away.

[AgainPsychoX]

It seems a workaround currently is to include an additional -- in the arguments to npm run when running on powershell.

Then it does not work in cmd or any other stuff, therefore its worse than it was (unless you are working alone on the local project and you want stick to powershell).

Also experiencing the issue, for now I will switch to using cmd.exe as shell, but it's painful sometimes.

[bottlezz]

It seems a workaround currently is to include an additional -- in the arguments to npm run when running on powershell.

Then it does not work in cmd or any other stuff, therefore its worse than it was (unless you are working alone on the local project and you want stick to powershell).

Also experiencing the issue, for now I will switch to using cmd.exe as shell, but it's painful sometimes.

triple dash works for me too "---" but still only powershell has this issue

[bruxisma]

Hello all, I've run into this issue again and I decided that I would get to the bottom of it and after a few hours of spelunking I've discovered the reason for why it fails in pwsh 7. I'll be putting an explicit detail for non-experts for the reason below and explain how I figured it out, and how it is failing, as it's a combination of several behaviors. Luckily, this can be fixed fairly easily and I should be able to open a PR to resolve this unless someone beats me to it. 🙂

So, strap in because this is going to be in depth. It's a small nightmare. Feel free to skip down to "the fix" if you don't care about the reasons. 🙂

The Explanation

To start with, pwsh itself treats -- special for its own syntax. This special character is treated like programs in other shells treat it: an "end of parameters" parameter. I first noticed something was up when I saw there is a slight difference in syntax highlighting in the terminal for this parameter. However this is barely noticeable if you have syntax highlighting enabled in your shell as the colors are simply the same as any other parameter but bold. I've attached a screenshot for sighted folks.

Notice how the -- is slightly brighter in the attached screenshot (I'm using Windows Terminal with a gruvbox theme). This same highlighting does not appear for other "special" parameters including the "stop parsing" --% token, which can't be used because when you run node in pwsh, it's executing a .ps1 file, and passing --% $args to node will of course error because it has no earthly idea what $args means.

This is further compounded by pwsh treating the execution of node as a native expression because of the & call operator, instead of something like Start-Process. Additionally because of rules regarding pwsh parsing command line arguments in a specific way is due to a result of how ProcessStart parses \, which in turn is a result of how the Win32 C function CommandLineArgvW parses command line arguments. If you're not aware, Win32 treats all arguments as a single string, and it is runtimes further up the chain that break them up into an array/list/sequence/etc.

So, when pwsh encounters --, it parses it out, like any other syntax. This brings us to the next issue: nopt, the node option parsing library. It only stops treating parameters as "done" once it sees a --, otherwise it parses all "valid" flags, shoves them into an object, and returns that. There is, at this point, no hope of retrieving or undoing the parse efforts to find these additional flags and they are removed from npm's CLI as a result and cannot be retrieved.

The Posit

So how do we get pwsh to ignore --? The answer is, in fact, terrible. If you simply use single quotes, you can "escape" the special parameter and it will be treated as normal: '--'. Yes, single quotes work just fine.Except, this only works on the command line. Attempting to use it inside of package.json will drop the --fix and then put quotes around -- directly. Instead, you must write \"--\" to properly escape the -- inside of package.json.

So how do we fix this? We can't let pwsh see the --. This means that if I have a variable $special = "--" and pass that in as a parameter, it will be passed through to npm correctly.

The Fix

My current plan for the fix is a hack but should be simple:

Check if -- was passed to the $MyInvocation.Line, find the parameter that came after it, and then insert a "--" into the $args array. We sadly can't detect the existence of a -- otherwise, but I'm confident we can at least work around this. 🙂

I'll be submitting a PR once I have a solution (and some tests!), so it might be a day or two before it's submitted. 😅

[bruxisma]

Brief update: While I have a working fix for just npm/npx, the issue is more to do with how cmd-shim currently generates the .ps1 file for npm. This is used to generate every .ps1 script, not just the NPM ones installed with node, but every "binary" installed by npm.

There may not be a fix for this without breaking the entire ecosystem beyond using --- for npm run (nopt parses any - of 2 or greater to be treated "as if" it were just -- on every platform) or waiting for Powershell 7.2 which has a new (breaking change) way of handling -- passed to native commands where they will pass through.

So much for my optimism regarding this being "easily" fixable 😔

[roman-balzer]

Hey, I do also currently have the same issue. I understand that you guys found out that the issue is with how Pwsh parses --. However, I noticed something else, I only have the issue with node version 17.1.0. Downgrading with nvm to 16.13 resolved this issue for me. I guess this has something to do with cmd-shim?.

Well guess i'll just use --- for now.

[kamranayub]

It was working for me with npm 6 and node 14, switched to npm 8 and node 16 and now it's not working. The --- is working for me for now though. Super goofy regression.

[jpierson-at-riis]

I've also run into this issue within my daily workflow after upgrading from node 12 to node 16 recently. We typically run cypress as an npm script as shown below.

npm run cypress:open -- --config baseUrl=https://dev.example.com

This had always worked prior without an issue with multiple versions of PowerShell on various systems. The main difference here seems to be the upgrade to node 16 that caused this regression. For now like others I'm using the --- workaround but I'm reluctant to add it to our internal documentation due to it likely hopefully being a temporary Windows only issue.

[hp8wvvvgnj6asjm7]

https://stackoverflow.com/questions/70090402/why-arent-npm-scripts-relaying-arguments-correctly-on-windows-powershell

[BrunoSMAlmeida]

Hey guys.
I need some help.
I try to run my code using this command:
npm run nameofthefile --env=uat
and the args return is undefined.
But, when I run npx nameofthefile --env=uat is return the env correctly.
Can someone help me?

[juna-an]

Hey everyone,
I confirm that the problem persists with npm v10.3.0. The workaround, that was mentioned above, of typing -- twice (for instance: npm run start -- -- --pkg hello) still works, but is not pretty.
Is there a plan to fix this at all?

[saschanaz]

So it's npm.ps1 that is broken?👀

[BurningEnlightenment]

@saschanaz as far as I understand it, npm.ps1 does nothing wrong except being a powershell script. PWSH preparses the commandline arguments for powershell scripts and discards the -- before it even reaches npm.ps1.

[goodov]

This is crazy. Why npm.ps1 even exists? What's wrong with launching npm via npm.cmd even from powershell?

Please just do the right thing and kill npm.ps1. No one expects such bug in the core part of node/npm infrastructure.

For anyone struggling with this, here's a quick one-liner to launch after node install:

Get-Command npm.ps1 | Select-Object -ExpandProperty Source | Remove-Item

[raky291]

Based on a Blog and Stack Overflow, we can use single quotes to pass arguments, it worked for me, example:

npm run script '--' --something

Blog: https://www.lloydatkinson.net/posts/2022/powershell-npm-scripts-and-silently-dropped-arguments/
Stack Overflow: https://stackoverflow.com/q/12197823/7034385